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LIST OF ACRONYMS 


Blue-Advantage Plus of Kansas City 
Behavioral Health Management Organization 
Consumer Assessment of Health Plans Survey 
Community CarePlus 

Centers for Disease Control and Prevention 


A statistical test that is used to examine the probability of a change or 
difference in rates is due to chance. 


Confidence Interval 
Children’s Mercy Family Health Partners 


Community Mental Health Center 


Centers for Medicare and Medicaid Services, U.S. Department of Health and 


Human Services 

Current Procedural Terminology 

Calendar Year 

U.S. Department of Health and Human Services 
Missouri Department of Health and Senior Services 
Division of Medical Services 

Missouri Department of Social Services 

Early, Periodic Screening, Diagnosis and Treatment 
External Quality Review 

External Quality Review Organization 
Fee-for-Service 

FirstGuard Health Plan 

HealthCare USA 


Healthy Children and Youth, the Missouri Medicaid EPSDT program 


Performance Management Solutions Group 


A division of Behavioral Health Concepts, Inc. 


HEDIS 
HIPAA 
HIS 


HMO 


ICD-9 


ICN 
ISCA 
LPHA 


MBE 


MC+ 


MC+ MCOs 
MCO 

MDI 

MHP 


MMIS 


MOHSAIC 


NCPDP 


NCQA 


N.S. 


NSF/CMS 1500 


PCP 


PIHP 


PIP 
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Health Plan Employer Data and Information Set 
Health Insurance Portability and Accountability Act 
Health Information Systems 

Health Maintenance Organization 


International Classification of Diseases, Ninth Revision, Clinical Modification, 
World Health Organization 


Internal Control Number 

Information Systems Capability Assessment 
Local Public Health Agency 
Minority-owned Business Enterprise 


The name of the Missouri Medicaid Program for families, children, and 
pregnant women. 


Missouri Medicaid Program Managed Care Organizations 
Managed Care Organization 

Missouri Department of Insurance 

Mercy MC+ 

Medicaid Management Information System 


Missouri Health Strategic Architectures and Information Cooperative, Missouri 
Department of Health and Senior Services Public Health Immunization Registry 


National Council for Prescription Drug Program 
National Committee for Quality Assurance 


Not significant, indicating that a statistical test does not result in the ability to 
conclude that a real effect exists. 


National Standard Format/ Center for Medicare and Medicaid Services Form 
1500 


Primary Care Provider 
Prepaid Inpatient Health Plan 


Performance Improvement Project 
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PRO Peer Review Organization 


QA & | Quality Assessment and Improvement Advisory Group 
QUUM Quality Improvement/Utilization Management Coordinator. 
Coordinator 

State Medicaid Agency, the Missouri Department of Social Services, Division of 
SMA 

Medical Services 

State Public Health Agency, the Missouri Department of Health and Senior 
SPHA 

Services 
UB-92 Universal Billing Form 92 
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GLOSSARY AND OPERATIONAL DEFINITIONS 





Administrative Method 


Accuracy (Match) Rate 


Accuracy of a data field 


Accuracy of the State 
encounter claims 


database 


Commission (or 
surplus encounter 


claim) 


Completeness of a 


data field 


Confidence interval or 


level 


The Administrative Method of calculating HEDIS Performance Measures 
requires the MCO to identify the denominator and numerator using 
transaction data or other administrative databases. The Administrative 
Method outlines the collection and calculation of a measure using only 
administrative data, including a description of the denominator (i.e., the 
entire eligible population), the numerator requirements (i.e., the 
indicated treatment or procedure) and any exclusions allowed for the 


measure. 


The ratio of identical or correct information in the medical record and 


the SMA relative to the number of encounters that took place. 


The extent to which an encounter claim field contains the correct type 
of information (e.g., numeric, alpha, alpha numeric) in the proper format 


(e.g., mm/dd/yyyy for date field). 


The extent to which encounters are being submitted for 100 percent of 


the services that are provided. | 


An encounter that is represented in the SMA encounter claims 


database but not the medical record; or a duplicate encounter. 


The extent to which an encounter claim field contains data (either 


present or absent). 


The range of accuracy of a population estimate obtained from a sample. 


' Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data: 


Second Edition 






Pontormsace Banagenant 
Solutions Group 
a Conc iol Rater amie core oe AC 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 


Xill 


Encounter data “Encounter data are records of health care services that have been 


provided to patients.” 2 


Error An error in coding or recording an encounter claim. 


Fault (Error) Rate The ratio of missing and erroneous records relative to the total 
number of encounters that took place?. The rate at which the SMA 
encounter claims data does not match the medical record or the MCO 


paid encounter claims data (the converse of match rate). 


Hybrid Method Hybrid Method requires the MCO to identify the numerator through 
both administrative and medical record data. The MCO reports a rate 
based on members in the sample who are found through either 
administrative or medical record data to have received the service 


identified in the numerator. 


Interrater reliability A method of addressing the internal validity of a study by ensuring that 
(IRR) data are collected in a consistent manner across data collectors. 
Omission (or missing An encounter that occurred but is not represented in the State 
encounter claim) encounter claims database. 

Paid claim An encounter claim that has been paid by the MCO. 


2 Medstat (1999).: A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data. 
Medstat: Santa Barbara. Second Edition 

3 Centers for Medicare and Medicaid Services (2002). Validating Encounter Data: A protocol for use in 
conducting Medicaid External Quality Review activities, Final Protocol, Version 1.0, U.S. Department of Health 
and Human Services. 
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Probability sample 


Random sample 


Reasonableness of a 


data field 


Reliability 


Sampling frame 


Sampling unit 


Simple sample 


Unpaid claim 


A sample in which every element in the sampling frame has a known, 
non-zero probability of being included ina sample. This produces 
unbiased estimates of population parameters that are linear functions of 


the observations from the sample data‘. 


Selection of sampling units from a sampling frame where each unit has 


an equal probability of selection. 


The extent to which an encounter claim field represents a valid value 
(e.g., an actual procedure code, actual birth date); also referred to as 


validity of the data. 


The consistency of findings across time, situations, or raters. 


The population of potential sampling units that meet the criteria for 
selection (e.g., Medical encounter claim types from January |, 2004 


through March 31, 2004). 


Each unit in the sampling frame (e.g., an encounter). 


Selection of sampling units from one sampling frame. 


All unpaid and denied claims from the MCO; All claims not paid by the 
MCO either through capitation or through other payment 


methodology. 


*Levy, P.S., Lemeshow, S. (1999). Sampling of Populations: Methods and Applications, Third Edition. J ohn 


Wiley and Sons: New York. 
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Figure 21 - Percentage Proportion of Claim Types per MC+ MCO, January |, 2005 — March 31, 


Figure 22 - Encounter Data Procedure Validation Rate, January |, 2005 — March 31, 2005 


Figure 23 - Encounter Data Diagnosis Validation Rate, January |, 2005 — March 31, 2005. 
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l.| Introduction 


The United States Department of Health and Human Services (DHHS), Centers for Medicare and 
Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid 
Managed Care programs by an External Quality Review Organization (EQRO). External Quality 
Review is the analysis and evaluation by an approved EQRO of aggregate information on quality, 
timeliness, and access to health care services furnished by Managed Care Organizations (MCOs) and 
their contractors to recipients of Medicaid managed care services. The Centers for Medicare and 
Medicaid Services (42 CFR §433 and §438; Medicaid Program, External Quality Review of Medicaid 
Managed Care Organizations) rule specifies the requirements for evaluation of Medicaid managed 
care programs. The present report summarizes the findings of the second year of implementation 
of the mandatory activities for External Quality Review of the MC+ Managed Care Program in 
Missouri as conducted by Behavioral Health Concepts, Inc., a PRO-Like Entity certified by CMS to 


conduct External Quality Review (EQR) in all U.S. states and territories. 


Four protocols were implemented: |) Validating Performance Improvement Projects;° 2) Validating 
Performance Measures;® 3) Validating Encounter Data;”? and 4) MCO Compliance with Managed 
Care Regulations. Each MC+ MCO conducted performance improvement projects (PIPs) during 
the 12 months preceding the audit; two of these PIPs were validated through a combination of self- 
selection and EQRO review. The final selection of PIPs to be audited was determined by the State 
Medicaid Agency (SMA; Missouri Department of Social Services, Division of Medical Services; DMS). 
The three performance measures validated were HEDIS 2005 measures of Well-Child Visits in the 
First 15 Months of Life, Childhood Immunization Status, Combination #2, and Annual Dental Visits. 


Validation of Encounter Data examined the completeness, accuracy, and reliability of specific fields in 


> Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Performance Improvement Projects: A protocol for use in Conducting Medicaid External Quality Review 
Activities, Final Protocol, Version 1.0, May 1, 2002. Washington, D.C.: Author. 

© Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Performance Measures: A protocol for use in Conducting Medicaid External Quality Review Activities, Final 
Protocol, Version 1.0, May 1, 2002. Washington, D.C.: Author. 

” Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Encounter Data: A protocol for use in Conducting Medicaid External Quality Review Activities, Final Protocol, 
Version 1.0, May 1, 2002. Washington, D.C.: Author. 

8 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR §400, 430, et al., Final 
Protocol, Version 1.0, February 11, 2003. Washington, D.C.: Author. 
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the SMA database; and the extent to which paid claims in the SMA were represented in the medical 
records of MC+ Managed Care Members. The EQRO conducted all protocol activities, with the 
exception of the MCO Compliance with Managed Care Regulations Protocol. The SMA conducted 


these activities and requested the EQRO to review them (Compliance Review Analysis). 


|.2 Preparation for the 2005 External Quality Review 


PREPARATION WITH THE STATE MEDICAID AGENCY 

Effective July |, 2005 the State of Missouri contract for the External Quality Review of the MC+ 
Managed Care Program (State of Missouri Contract No: C301 154001, Amendment No.: 006) was 
revised to comply with federal requirements for states to contract with an external, independent 
entity to implement the mandatory protocols for External Quality Review. The first monthly 
meeting for planning the scope of work, technical methods and objectives, and analyses was held by 
the SMA on September 22, 2005. Meetings were held with the SMA and the EQRO on October 27, 
2005, November |5, 2005, December 15, 2005, January 9, 2006, February 21, 2006, March 27, 2006, 
and April 24, 2006. Additional meetings and teleconference calls were conducted as needed 


between SMA and EQRO personnel. 


At the first meeting in September 2005, the previous years’ report was discussed, the new EQRO 
Project Director was introduced to the SMA, and the plan for the 2005 audit was discussed. During 
the month of September, the EQRO clarified the SMA’s objectives for each of the protocols, 
developed data requests, prepared detailed proposals for the implementation and analysis of data for 
each protocol, and prepared materials for SMA review. Written proposals for each protocol were 
submitted on October | 1, 2005 by the EQRO for review, discussion, revision, and approval. By 
November 7, 2005, the EQRO had negotiated with the SMA the data request for State encounter 
data to be validated. All protocols were revised, finalized and submitted to the SMA by October 
2005. 


PREPARATION OF MC+ MCOs 
During October 2005, preparation of MC+ MCOs for the implementation of the 2005 EQRO was 
conducted by the EQRO Project Director and personnel. To begin, the EQRO Project Director 


presented a timeline for project implementation and answered MCO questions at the October 20, 
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2005 MC+ Managed Care All-Plan Meeting. The EQRO Project Director and personnel then 


conducted orientation to the protocols and the EQRO processes with each MC+ MCO. 


The EQRO Assistant Project Director arranged the dates of the teleconference calls with MC+ 
MCO QI/UM Coordinators or Medicaid Plan Administrators. A detailed presentation, tentative list 
of data requests, and the proposals approved by the SMA were sent to MC+ MCOQOs prior to the 
teleconference orientation sessions. MC+ MCOs were requested to have all personnel involved in 
fulfilling the requests or in implementing activities related to the protocols (e.g., performance 
improvement projects to be validated, performance measures to be validated, encounter data 
requested) present at the teleconference calls. [The orientation presentation is contained in 
Appendix |.] An SMA representative attended all conference calls and received minutes of the 
meetings taken by the EQRO upon completion of all the calls. Conference calls with EQRO and 
MC+ MCO personnel occurred between October 21, 2005 and October 31, 2005. To avoid 
confusion and the inundation of multiple requests at once, the requests for information from MC+ 
MCOs were implemented in a staged approach from November |, 2005 through December 9, 2005. 
All communications (letters, general and specific instructions) were submitted for review, revision, 


and approval by the SMA prior to sending them to the MC+ MCOs. 


DEVELOPMENT OF WORKSHEETS, TOOLS, AND RATING CRITERIA 

The EQRO Project Director, Research Associate, Assistant Project Director, a health services 
researcher, and a healthcare provider were responsible for modifying the worksheets and tools used 
by the EQRO during the 2004 audit. The EQRO Assistant Project Director revised the worksheet 
(Attachment B) of the Validating Performance Improvement Project Protocol to add detail for 


several items that were specific to the MC+ Managed Care Program. 


For the Validating Encounter Data Protocol, the EQRO Project Director revised both the data 
analytic plan in collaboration with the SMA as well as methods and procedures based on the 
content, quality and format of data provided by the SMA and MC+ MCOs. The SMA selected the 
fields to validate for completeness, accuracy, and reliability of paid claims submitted by MC+ MCOs. 
The EQRO developed definitions of all field parameters for review, revision, and approval by the 
SMA. Encounter data critical field parameters were approved by the SMA at the December |[5, 
2005 meeting between the SMA and the EQRO. 


(x aa, seen Performance Management Solutions Group 
a A division of Behavioral Health Concepts, Inc. 


Missouri MC+ Managed Care External Quality Review Section | 
Report of Findings — 2005 Executive Summary 


The Validating Performance Measures Protocol worksheets were revised and updated by the EQRO 
Project Director to reflect the Performance Measures selected for review for HEDIS 2005. The 
worksheets had been developed by Behavioral Health Concepts, Inc. staff during the previous year’s 


audit. 


The SMA had already conducted the activities of the MC+ MCO Compliance with Managed Care 
Regulations Protocol through the state contract compliance monitoring process and the work of the 
EQRO involved the review and evaluation of this information (see Medicaid Program; External 
Quality Review of Medicaid Managed Care Organizations of 2003, CFR §438.58). The state contract 
for EQRO requires the review of SMA’s activities with regard to the Protocol, however, additional 
policies and documents were requested prior to and during the on-site visits with MC+ MCOs 
when information was incomplete or unclear. To facilitate the review of compliance with federal 
regulations, the EQRO Assistant Project Director revised a previously developed cross-walk 
between the SMA contract requirements for Medicaid managed care and the federal Medicaid 


Managed Care Regulations. 


The MC+ Managed Care Program consultant, who has participated in the EQRO for the past three 
years, reviewed and refined the tool. Feedback on inconsistencies between the Medicaid Managed 
Care contract and federal requirements was provided immediately to the SMA. The EQRO utilized 
the rating system developed during the 2004 audit to provide ratings for each MCOs’ compliance. 
The SMA provided state compliance review information to the EQRO for all MC+ MCOs in 
October 2005. The EQRO staff and the consultant reviewed all available materials and met with 
SMA staff on October 27, 2005 to clarify SMA comments and compliance ratings; and identify issues 
for follow-up at site visits. Updates on MC+ MCO compliance were provided through early 
February 2006 to ensure that the EQRO had up-to-date information prior to the beginning of the 
on-site reviews. Recommended ratings were provided to SMA on April 24, 2006, which were 


approved for utilization in this report. 


The following sections summarize the aggregate findings and conclusions for each of the mandatory 
protocols. The full report is organized according to each protocol and contains detailed 
descriptions of the technical methods, objectives, findings, and conclusions (strengths, areas for 
improvement, and recommendations). In addition, it provides MCO to MCO comparisons and 


individual MC+ MCO summaries for each protocol. 
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1.3 Validation of Performance Improvement Projects 


For the Validating Performance Improvement Projects (PIP) Protocol, the EQRO validated two PIPs 
for each MCO that were underway during the previous |2 month period at each MC+ MCO, for a 
total of 14 PIPs validated. Eligible PIPs for validation were identified by the MC+ MCOs, SMA, and 
the EQRO. The final selection of the PIPs for the 2005 validation process was made by the SMA in 
October 2005. PIPs are to be aimed at studying the effectiveness of clinical or non-clinical 
interventions, and should improve processes highly associated with healthcare outcomes, and/or 
healthcare outcomes themselves. They are to be carried out over multiple re-measurement periods 
to measure: |) improvement; 2) the need for continued improvement; 3)or stability in 
improvement as a result of an intervention. Under the State contract for Medicaid Managed Care, 
MC+ MCOs are required to have two active PIPs, one of which is clinical in nature and one non- 
clinical. Specific feedback and technical assistance was provided to each MC+ MCO by the EQRO 


during the site visits for improving study methods, data collection, and analysis. 


STRENGTHS 


|. There was good topic identification and good intervention development for PIPs. MC+ MCO 
personnel were well qualified to identify study topics, areas in need of improvement, and 
interventions to address barriers to quality of care and health outcomes. Interventions were 
aimed at key aspects of enrollee care and services. 


2. All aspects of the PIPs indicated significant improvement in the structure, intent, and depth of 
the process utilized to produce these studies. These studies incorporated many of the 
requirements of the Conducting Performance Improvement Projects protocol. The MC+ 
MCOs clearly intended to utilize improved methods of producing health care services into their 
day-to-day operations at the completion of the PIP when appropriate. 


3. A number of the PIPS (Pre-Authorization Improvement, ADHD, Increased Blood Lead Level 
Testing at 12 & 24 Months, Access to Primary Care Services, Improving Lead Screening at 6-36 
Months of Age, Improving Asthma Medication Management) were judged to be likely or highly 
likely to produce credible and valid findings to identify Best Practices. 


4. MC+ MCO personnel were attentive and responsive to technical assistance for the 
implementation of PIPs and presented goals or plans for improving the process in the future. 
MC+ MCOS instituted personnel and/or role changes to improve the focus and expertise of 
personnel responsible for PIP implementation. 
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AREAS FOR IMPROVEMENT 


Study questions should continue to be refined. In some cases the study question did not frame 
the purpose, intended outcomes, goals, and methods for the study. It is critical that there is a 
link between the study questions, measures of implementation, measures of outcomes, and 
analysis of findings to address the problems or issues identified. 


Data analysis plans were not described in a number of the PIPs. The method of data collection 
and compilation is often dependent on the plan for data analysis. A plan should be developed 
prior to conducting a PIP, and modified as needed to be able to test for significant improvement 
or stability of performance on the outcome measures over time. Data analyses could not be 
evaluated without a description of the plan. 


There were PIPs underway or ongoing that resulted in the potential for credible findings. 
Ensuring that the project is started early enough to provide some data and data analysis is 
essential in completing the validation process. It appeared that many MC+ MCOs conduct PIPs 
on an ongoing basis as part of their quality improvement program. Continuing to utilize these 
PIPs as tools to improve the organizations ability to serve members will be beneficial. 


RECOMMENDATIONS 
I. 





It is recommended that MC+ MCOs obtain additional training, assistance and expertise for the 
design, statistical analysis, and interpretation of PIP findings. One MC+ MCO (Children’s Mercy 
Family Health Partners) utilized the services of a statistician from a local university to ensure 
valid and reliable findings. 


In the design of PIPs, MC+ MCOs need to use generally accepted practices for program 
evaluation to conduct PIPs. In addition to training on the development of PIPs and on-site 
technical assistance, references to the CMS protocol, “Conducting Performance Improvement 
Projects” were recommended by the EQRO at each MC+ MCO as a guideline to frame the 
development, reporting and analysis of the PIP. 


lt is recommended that a statewide PIP be identified by the SMA and the MC+ QA & | Group 
for planning and implementation one year prior to the planned implementation. The topic 
should be chosen by taking into account the findings of this report and the current topics 
chosen by the MC+ MCOs. Topics that are likely to affect a broad segment of the population 
and key healthcare outcomes should also be considered. 
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1.4 Validation of Performance Measures 


The Validating Performance Measures Protocol requires the validation or calculation of three 
performance measures at each MC+ MCO by the EQRO. The measures selected for validation by 
the SMA are required to be submitted by each MC+ MCO on an annual basis. The measures were 
also submitted by the State Public Health Agency (SPHA; Missouri Department of Health and Senior 
Services; DHSS) for all Health Maintenance Organizations (HMOs) operating in the State of 
Missouri. They were: |) HEDIS 2005 Childhood Immunization Status, Combination #2; 2) HEDIS 
2005 Well-Child Visits in the First 15 Months of Life ; and 3) HEDIS 2005 Annual Dental Visit. 
Detailed specifications for the calculation of these measures were developed by the National 
Committee for Quality Assurance (NCQA), a national accrediting organization for managed care 
organizations. The EQRO examined the information systems, detailed algorithms, MC+ MCO 
extract files, medical records, and data submissions provided to the SPHA to conduct the validation 
activities of this protocol. The data reported to the SPHA was based on MC+ MCO performance 
during 2004. MC+ MCOs were given an opportunity to review and correct the data presented to 
the SPHA; and were provided with the opportunity to review EQRO findings for comment and 


correction. 


STRENGTHS 


|. MC+MCOs have strong management information systems for the documentation and payment 
of services to providers and the tracking of member information. These systems and the 
processes for performance measure calculation are well documented and MC+ MCOs retain 
qualified personnel for the programming of data specifications to calculate performance 
measures. 


2. Six of the seven MC+ MCOQOs incorporated external data (State Public Health Immunization 
Registry; the Missouri Health Strategic Architecture and Information Cooperative; MOHSAIC) 
to capture immunizations delivered outside the MC+ MCO for the HEDIS 2005 Childhood 
Immunization Status, Combination #2. 


3. Six of the seven MC+ MCOs produced the HEDIS 2005 Annual Dental Visits Measure in a 
manner that was Fully Compliant with the specifications. 


4. One MC+ MCO, that was bound to be Substantially Compliant with the calculation of the 
HEDIS 2005 Childhood Immunization Status, Combination # 2 measure, had a higher rate than 
the National Medicaid rate for this measure. 


5. One MC+ MCO that was Substantially Compliant with the calculation of the HEDIS 2005 Well- 
Child Visits in the First 15 Months of Life Measure had a rate higher than the National 
Commercial and Medicaid rates for this measure. 
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6. Four of the seven MC+ MCOs employed NCQA-certified software for the rate calculation 
process, which requires passing test file procedures by NCQA for accurate and valid 
programming and calculation of the specifications. 


7. In response to the EQRO exit interview comments and preliminary findings for this Protocol, 
several MC+ MCOs developed corrective action plans for improving the process of 
documentation, rate calculation, oversight, and staff training for the calculation of performance 
measures. 


AREAS FOR IMPROVEMENT 


|. The HEDIS 2004 Childhood Immunization Status, Combination #2 measure was unable to be 
validated for three of the seven MC+ MCOs and does not represent a valid measure of 
performance for the MC+ Managed Care Program. 


2. In many cases, there was limited organizational knowledge of the process of calculating 
measures. This was related to the use of vendors for various aspects of claims administration 
and the rate calculation process. 


3. There continues to be variability in the implementation of the Hybrid Method. The EQR 
validation used the same criteria for numerator events across all MC+ MCOs in the validation 
process. In addition, not all MC+ MCOs reviewed all of the medical records sampled for the 
Hybrid Method of calculation. 


RECOMMENDATIONS 


|. The SMA should use the results of the EQR, as much as possible, to build and negotiate rate 
increases for Managed Care Organizations in Missouri. 


2. The SMA and SPHA should continue to support efforts to improve the utility and functionality 
of the State Public Health Immunization Registry (MOHSAIC) as well as encourage public, 
private, and non-profit providers of immunizations to use the Registry so as to obtain complete 
information about the level of care provided to MC+ Managed Care Members for the 
administration of immunizations. 


3. For the calculation of the HEDIS Childhood Immunization Status, Combination #2 measure, the 
Hybrid Method and the incorporation of MOHSAIC data should be required by the SMA to 
facilitate accurate and valid MC+ MCO comparisons and a valid statewide rate for comparison 
of performance with other states. 


4. MC+MCOs should document policies, procedures, processes, and responsibilities of personnel 
and vendors involved in calculating and reporting HEDIS measures for the MC+ Managed Care 
Program. This should include vendor oversight and review of files and data produced to 
calculate the measures. 
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1.5 Encounter Data Validation 


Encounter claims data are used by SMAs to conduct rate setting and quality improvement 
evaluation. Before SMA encounter claims data can be used, it is necessary to establish the extent to 
which the data for critical fields (e.g., diagnosis and procedure codes, units and dates of service, 
member and provider identifiers) are complete (each field contains information), accurate (the 
information contained in each field is of the right size and type), and valid (the information 
represents actual dates or procedure and diagnosis codes). Several critical fields for each of six 
claim types (Medical, Dental, Home Health, Inpatient, Outpatient, Hospital, and Pharmacy) were 
identified by the SMA and examined by the EQRO for completeness, accuracy, and validity using an 
extract file from SMA paid encounter claims. To examine the extent to which the SMA encounter 
claims database was complete (the extent to which SMA encounter claims database represents all 
claims paid by MC+ MCOQs); the level and consistency of services was evaluated by examining the 
rate of each of six claim types. Additionally, the representativeness (or completeness) of the SMA 
encounter claims database was examined by comparing data in the SMA encounter claims database 
to the medical records of members. A random sample of medical records was used to compare the 
diagnosis and procedure codes in the SMA encounter claims database with documentation in MC+ 
member medical records. The findings of these comparisons were used to determine the 
completeness of the SMA encounter claims database in regards to the medical records of members. 
The completeness of the SMA paid encounter claims was then compared with MC+ MCO records 
of paid and unpaid claims. This proved to be a difficult task, as all of the MC+ MCO data submissions 
did not include unique claim identifiers that could be used to accomplish this comparison. Only two 
of the seven MC+ MCOs provided data in the format necessary to make the comparisons; the 
results obtained are detailed in the results of the Aggregate Encounter Data Validation section of 


this report. 


STRENGTHS 


|. The majority of critical fields evaluated for each of the six encounter claim types were accurate, 
complete, and valid. 


2. The Pharmacy, Dental, and Home Health critical fields contained valid data for the analysis of 
paid encounter claims. 
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AREAS FOR IMPROVEMENT 


Medical records that did not have diagnosis or procedure codes that matched those in the SMA 
encounter claims database were in error primarily due to missing or illegible data. 


2. Only two MC+ MCOs were able to produce paid and unpaid claims in valid formats for 
encounter data validation to assess the completeness of paid claims as represented in the SMA 
encounter claims database or to identify omission and commission errors related to encounter 
data submission. 

3. All MC+ MCOs had invalid coding in the Medical Claim Types. 

4. For all MC+ MCOs invalid dates were found in the Inpatient Claim Type. 

5. The volume and consistency of services across MC+ MCOs and claim types were highly variable, 
with no patterns across MC+ Managed Care Regions observed. 

RECOMMENDATIONS 

|. MC+MCOs should explore potential reasons for variation in claim types and in the proportion 
of each claim type to all claim types. 

2. MC+MCOs should examine and revise as needed internal system edits for invalid procedure 
codes in the NSF/CMS 1500 file layout for the Outpatient Procedure Code and run validity 
checks after the programming of new edits. 

3. For the Inpatient claim type (UB-92 file layout), improve the rate of valid Discharge Dates to flag 
invalid entries of “99999999 and blank entries for the Units of Service field. Error checks for 
the Diagnosis Code field should also be conducted to ensure no blank fields. 

4. It is recommended that MC+ MCOs emphasize to providers the importance of documentation 
of diagnostic and procedural code information in the medical record. 

5. MC+MCOQOs should include State issued ICN numbers (in the format issued by the State) in all 





data submissions to the EQR, this would allow more accurate matching of encounters between 
the MC+ MCO and SMA extract files. 
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1.6 MC+ MCO Compliance with Managed Care Regulations 


The purpose of the protocol to monitor MCO Compliance with Managed Care Regulations is to 
provide an independent review of MC+ MCO activities and assess the outcomes of timeliness and 
access to the services provided by the MC+ MCOs. The protocol requires the utilization of two 
main sources of information to determine compliance with federal regulations. These sources of 
information are document review and interviews with MC+ MCO personnel. This combination of 
information was designed to provide the SMA with a better understanding of organizational 


performance at each MC+ MCO. 


The policy and practice in the operation of each MC+ MCO was evaluated against the seventy (/0) 
regulations related to operating a Medicaid managed care program. The regulations were grouped 
into three main categories: Enrollee Rights and Protections, Quality Assessment and Improvement, 
and Grievance Systems. The category of Quality Assessment and Improvement was subdivided into 
three subcategories: Access Standards, Structure and Operation Standards, and Measurement and 
Improvement. Initially, the SMA reviewed each MC+ MCOs’ policy to determine compliance with 
the requirements of the MC+ Medicaid Managed Care Contract. These determinations and their 
application to the requirements of the federal regulations were assessed by the EQRO. The EQRO 
also focused on follow up to the findings reported in the 2004 report by concentrating efforts of 
technical assistance and assessment on the items that were rated “Not Met” in 2004. Additional 
document review occurred when the MC+ MCO policy submission did not meet MC+ Medicaid 
Managed Care contract requirements, or where clarification was necessary. A set of interview 
questions specific to each MC+ MCO was developed to elicit information that validated 


organizational practice and explored issues not fully addressed in the documents. 


STRENGTHS 


|. Two of the MC+ MCOs “met” (100%) applicable federal regulations and State compliance 
requirements. There were no regulations rated as “Unmet” for any of the MC+ MCOs. The 
ratings for every section of the protocol indicated an improvement in all areas of policy and 
procedure submission and approval, as well as practice. 


2. The MC+ MCOs were aware of their need to provide quality services to members in a timely 
and effective manner. VVhere there were issues with access to services, the MC+ MCOs 
responded quickly and effectively. 
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MC+ MCOs remained invested in developing programs and providing services beyond the strict 
obligations of the contracts. Preventive health and screening initiatives exhibited a commitment 
to providing the best healthcare in the least invasive manner to their MC+ Members. 
Partnerships with local universities and medical schools provided opportunities to obtain 
cutting-edge and occasionally experimental treatment options, which would not otherwise be 
available to MC+ Members. 


Across all MC+ MCOs there was recognition of the importance of being in compliance with the 
federal regulations. The MC+ MCOs exhibited significant improvement in their attention to the 
details of producing effective and required policies and procedures, and creating an environment 
that rewards adherence to these policies. 


AREAS FOR IMPROVEMENT 


All MC+ MCOs did not have their written policies and procedures completed to ensure that a 
consistent application of contractual requirements and federal regulations was occurring. 


MC+ MCOs continued to struggle with recruitment of certain specialty physicians. Throughout 
discussions with MC+ MCOQOs the lack of orthopedic surgeons, neurosurgeons and 
child/adolescent psychiatrists was identified as a problem. The MC+ MCOs have made 
accommodations to ensure that members received the services required. Through the use of 
advance practice nurses, cooperative agreements with medical schools, and the willingness to 
reimburse at commercial insurance rates, the MC+ MCOs attempt to ensure that members 
have access to these services. MC+ MCOs expressed continued concern for improvement in 
this area. 


The use of data for quality improvement purposes and examination of healthcare outcomes was 
just beginning. Continued growth in the utilization of all of the data available to drive healthcare 
practice and initiatives is required to improve quality and access to care. 


RECOMMENDATIONS 
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MC+ MCOs must continue to recognize the need for timely submission of all required policy 
and procedures. The majority of the MC+ MCOs put a tracking or monitoring system into 
place to ensure timely submission of documentation requiring annual approval. These systems 
must be maintained to ensure that this process remains a priority for all MC+ MCOs. 


MC+ MCOs identified the need for continuing to monitor provider availability in their own 
networks. Although most MC+ MCOs had the number of primary care physicians (PCPs) and 
specialists required to operate, they admitted that many of these PCPs had closed panels and 
would not accept new patients. Ensuring that there is adequate access for all members, 
including new members, should be a priority for all MC+ MCOs. 


MC+ MCOs identified improvement in their Quality Assessment and Improvement programs, 
and how this enhanced their ability to provide adequate and effective services to members. 
These efforts must be relentlessly continued to ensure that the organizations remain aware of 
areas for growth and improvement. These efforts ensure that the quality, timeliness and access 
to care required for member services is maintained at an exceptional level. 
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2.0 VALIDATION OF PERFORMANCE IMPROVEMENT 


PROJECTS 





2.1 Definition 


A Performance Improvement Project (PIP) is defined by the Center for Medicare and 
Medicaid Services (CMS) as “a project designed to assess and improve processes, and 
outcomes of care...that is designed, conducted and reported in a methodologically sound 
manner.” The Validating Performance Improvement Projects Protocol specifies that the 
EQRO conduct three activities in the validation of two PIPs at each MCO that have been 
initiated, are underway, were completed during the reporting year, or some combination of 
these three stages. The State Medicaid Agency (SMA: the Department of Social Services, 
Division of Medical Services: DMS) elected to examine projects that were underway during 
the preceding 12 months. Criteria for identification of a PIP as outlined in the CMS 


protocols include the following: 


= — PIPs need to have a pre-test, intervention, and post-test 

= — PIPs need to control for extraneous factors 

= —_PIPs need to include an entire population 

# Pilot projects do not constitute a PIP 

» — Satisfaction studies alone do not constitute a PIP 

= Focused studies are not PIPs: A focused study is designed to assess processes and outcomes on one-time basis, 
while a PIP is to improve processes and outcomes of care over time. 


The State of Missouri contract for Medicaid Managed Care (RFP No. B3Z03182, Contract 
Amendment 002, 08/25/2003) describes the following requirements for MC+ MCOQs in 


conducting PIPs: 


Performance Improvement Projects: The health plan must conduct performance improvement 

projects that are designed to achieve, through ongoing measurements and intervention, significant 

improvement, sustained over time, in clinical care and nonclinical care areas that are expected to have 

a favorable effect on health outcomes and member satisfaction. The health plan must report the status 

and results of each project to the state agency as requested. The performance improvement projects 

must involve the following: 

Measurement of performance using objective quality indicators. 

Implementation of system interventions to achieve improvement in quality. 

Evaluation of the effectiveness of the interventions. 

Planning and initiation of activities for increasing or sustaining improvement. 

Completion of the performance improvement project in a reasonable time period so as to 

generally allow information on the success of performance improvement projects in the 

aggregate to produce new information on quality of care every year. 

e Performance measures and topics for performance improvement projects specified by CMS in 
consultation with the state agency and other stakeholders. 
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2.2 Purpose and Objectives 


The purpose and objectives of the present review were to evaluate the soundness and 
results of PIPs implemented by MC+ MCOs in 2005. The MC+ MCOs were to have two 
active PIPs in place, one clinical and one nonclinical. The validation process examines the 


stability and variability in change over multiple years. 


2.3 Technical Methods 


There are three evaluation activities specified in the protocol for Validating Performance 


Improvement Projects. Activity One consists of ten steps: 


Activity One: Assessing the MCOs /PIHP’s Methodology for 
Conducting the PIP 


|. Step One: Review the selected study topic(s) 

Step Two: Review the study question(s) 

Step Three: Review selected study indicator(s) 

Step Four: Review the identified study population 

Step Five: Review sampling methods (if sampling was used) 
Step Six: Review the MCOs/PIHP’s data collection procedures 
Step Seven: Assess the MCOs/PIHP’s improvement strategies 


Step Eight: Review data analysis and interpretation of study results 


2, 
3. 
4. 
5. 
6. 
Te 
8. 
9. 


he 


Step Nine: Assess the likelihood that reported improvement is “rea 
improvement 
. Step Ten: Assess whether the MCO/PIHP has sustained its documented 


improvement 





Activity Two (Verifying PIP Study Findings) is optional, and involves auditing PIP data. Activity Three 
(Evaluate Overall Reliability and Validity of Study Findings) involves the judgment of whether the 
results and conclusions drawn from the PIP are valid and reliable. Activities One and Three were 


conducted by the EQRO. 
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TIME FRAME AND SELECTION 

Two projects that were underway during the preceding |2 months at each MC+ MCO were 
selected for validation. The projects to be validated were reviewed with SMA and EQRO staff, in 
October 2005. The intent was to identify projects which were mature enough for validation (i.e., 
planned and in the initial stages of implementation), yet still underway during calendar year 2005. 
The SMA made the final decision regarding the actual PIPs to be validated from the descriptions 
submitted by the MC+ MCOs. 


PREPARATION OF MC+ MCOs 

All MC+ MCOs were contacted during October 2005 to prepare them for the 2005 External 
Quality Review. All MC+ MCO quality management staff or plan administrators were contacted to 
discuss the onset of External Quality Review Organization (EQRO) activities and to schedule 
training teleconferences for later in October. The MCOs were explicitly requested to have their 
staff or subcontractors available who would be responsible for obtaining and submitting the data 
required to complete all validation processes. During these teleconferences, all aspects of the EQR, 
including the requirements of submissions for the Performance Improvement Projects, were 


discussed. 


The training teleconference agenda, methods and objectives, and schedule were sent to all MC+ 
MCOs, with approval from the State Medicaid Agency (SMA), on October 18, 2005. SMA staff 
agreed to participate in these conference calls, which were held in late October, allowing time for 
presentation of information, clarification, and questions. The original submission of data was 
scheduled prior to the end of the calendar year, which was prior to the completion of most of the 
projects to be submitted for validation. As a result additional information was accepted at the on- 


site reviews, or shortly afterward. 
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REVIEWERS 

Three reviewers conducted the Validating Performance Improvement Project Protocol activities, 
including interviews and document review. The External Quality Review Organization (EQRO) 
Project Director is a licensed attorney with a graduate degree in Health Care Administration, and 
five years of experience in public health and managed care in two states. She conducted interviews 
and provided oversight to the PIP Protocol team. The Assistant Project Director was conducting 
her second review. She had experience with the MC+ Managed Care Program implementation and 
operations, interviewing, program analysis, and Medicaid managed care programs in other states, and 
ten years experience in program evaluation and research. The second reviewer participated in four 
previous MC+ Managed Care Program EQRs and on-site visits. This reviewer was knowledgeable 
about the MC+ Managed Care Program through her experience as a former SMA employee 
responsible for quality assessment and improvements, as an RN, and a consultant. All reviewers 
were familiar with the program improvement project requirements and validation process, as well as 


research methods, and the requirements of the MC+ Managed Care Program. 


2.4 Procedures for Data Collection 


The evaluation involved review of all materials submitted by the MC+ MCOs including, but not 
limited to, the materials listed below. During the training teleconferences MC+ MCOs were 
encouraged to review Attachment B of the Validating Performance Improvement Projects Protocol 
and ensure that they include information documents, tools, and other information necessary to 


evaluate the projects submitted, based on this tool. 


» — Narrative descriptions 

= Problem identification 

= Hypotheses 

= Study questions 

# Description of interventions(s) 

=» Methods of sampling 

# — Planned analysis 

» — Sample tools, measures, survey, etc. 

= Baseline data source and data 

» Cover letter with clarifying information 
= Overall analysis of the validity and reliability of each study 
= Evaluation of the results of the PIPs 


The EQRO Project Director, Assistant Project Director, and review consultant met with the MC+ 
MCO staff responsible for planning, conducting, and interpreting the findings of the PIPs during the 


on-site reviews occurring between February 28, 2006 and March 15, 2006. The review focused on 
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the findings of projects conducted during 2005. MC+ MCOs were instructed that additional 
information and data not available at the time of the original submission could be provided at the 
time of the on-site review or shortly afterwards. The time scheduled during the on-site review was 
utilized to conduct follow-up questions, to review data obtained, and to provide technical assistance 
to MC+ MCOs regarding the planning, implementation and credibility of findings from PIPs. In 
addition individual clarifying questions were used to gather more information regarding the PIPs 
during the on-site interviews. The following questions were formulated and answered in the original 


documentation, or were posed to the MC+ MCOs during the on-site review: 


» Who was the project leader? 

= How was the topic identified? 

= How was the study question determined? 

=» What were the findings? 

=» What were the interventions(s)? 

=» What was the time period of the study? 

» Was the intervention effective? 

» What did the MC+ MCO want to learn from the study? 


All PIPs were evaluated by the Review Consultant and the Assistant Project Director. In addition, 
the projects were reviewed with follow-up suggestions posed by the Project Director, who 


approved final ratings based on all information available to the team. 


ANALYSIS 

All PIPs submitted by MC+ MCOs prior to the site visits were reviewed using an expanded version 
of the checklist for conducting Activity One, Steps | through 10, and Activity Three (Judgment of 
the Validity and Reliability of the PIPs) of the Validating Performance Improvement Projects 
Protocol, Attachment B (see Appendix 2). Because specific criteria may not have been applicable 
for projects that were underway at the time of the review, some specific items were considered as 
“Not Applicable.” Criteria were rated as “Met” if the item was applicable to the PIP, if there was 
documentation addressing the item, and if the item could be deemed Met based on the study design. 
The proportion of items rated as “Met” was compared to the total number of items that were 
applicable for the particular PIP. Given that some PIPS were underway in the first year of 
implementation, it was not possible to judge or interpret results, validity of improvement, or 
sustained improvements (Steps 8-10). The final evaluation of the validity and reliability of studies 
underway were based on the potential for the studies to produce credible findings. Detailed 
recommendations and suggestions for improvement were made for each item where appropriate, 


and are presented in the individual MC+ MCO summaries. Some items are rated as ““Met’’ but 
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continue to include suggestions and recommendations as a method of improving the information 
presented. The following are the general definitions of the ratings developed for evaluating the PIPs. 
Met: Credible, reliable, and valid methods for the item were documented. 


Partially Met : Credible, reliable, or valid methods were implied or able to be established for part of the 
item. 


Not Met: The study did not provide enough documentation to determine whether credible, reliable, 


methods were employed; errors in logic were noted; or contradictory information was 
presented or interpreted erroneously. 


Not Applicable: Only to be used in Step 5, when there is clear indication that the entire population was 
included in the study and no sampling was conducted; or in Steps 8 through 10 when the 
study period was underway for the first year. 





2.5 Findings 


Below are the PIPs identified for validation at each MC+ MCO. 
Community CarePlus Emergency Room Utilization (Children Receiving ER Services at 


Cardinal Glennon Hospital) 


Early Intervention in Prenatal Care Management and the 
Relationship to the Very Low Birth Weight Babies 
Mercy MC+ Medicaid Emergency Room Utilization 


Customer Service — Member Services Call Center 
HealthCare USA 2004/2005 Pre-Authorization Improvement Project 
2005 Lead Performance Improvement Project 


Missouri Care Attention Deficit Hyperactivity Disorder Performance 
Improvement Project 


Lead Performance Improvement Project 


Children’s Mercy Family Access to Primary Care Services 


Health Partners Improving Lead Screening at 6-36 Months of Age 
FirstGuard Health Plan Improving Birth Weight Outcomes 

Improving Asthma Medication Management 
Blue Advantage Plus of Lead Testing Improvement Project 


Kansas City Improving Care for Asthmatics 
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STEP 1: SELECTED STUDY TOPICS 

Study topics were selected through data collection and the analysis of comprehensive aspects of 
member needs, care, and services. Study topics intended to address a broad spectrum of key aspects 
of member care and services. In all cases they included all enrolled populations pertinent to the 
study topic without excluding certain members. Four of the |4 PIPs addressed blood lead level 
testing; two addressed care for members with asthma; two addressed access to care for pregnant 
members with the goal of reducing low birth weigh infants; one addressed ADHD treatment; one 
addressed access to primary care services; two addressed emergency room utilization; one 
addressed improving the pre-authorization process: and one addressed customer call center 
services. Table | shows the ratings for each item and PIP by MC+ MCO. All 14 PIPs provided some 
rationale demonstrating the extent of the need for the PIP. Many discussed literature supporting 
the activities to be undertaken, and provided some benchmark comparison data. While this entire 
section was not perfect the MC+ MCOs met all the criteria required 85.7% of the time. MCt+ 
MCOs addressed a broad spectrum of key aspects of member care and services (12 of the |4 PIPs, 
85.7%, Met this criteria and 2 Partially Met this criteria (Community Care Plus PIPs); Step 1.2). 
Clinical and nonclinical interventions were identified and aspects of enrollee care and services that 
were related to the identified problem were described. Utilization or cost issues may be examined 
through a PIP, but should not be the sole focus of the study. There were some descriptions of the 
member populations targeted for intervention in the PIPs. Because the MC+ MCOQOs vary widely in 
the member populations they serve (e.g., other state Medicaid managed care members, commercial 
members, or Medicare members), it was not possible to determine the extent to which the PIP 
identified, addressed, and measured the needs of the MC+ Managed Care Program population in all 
cases. In addition, PIPs should specifically indicate whether all enrolled populations within the MC+ 
Managed Care Program were included in the interventions. Finally, age and demographic 


characteristics should be described. Seven of the I|3 PIPs (53.8%) Met these criteria (Step I.3). 
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STEP 2: STUDY QUESTIONS 

Study questions are statements in the form of a question that describe the potential relationship 
between the intervention, the intended outcome, and the data to be obtained and analyzed. They 
should be specific enough to suggest the study methods and the outcome measures. The MCt+ 
MCOs made a concerted effort to ensure that statements were provided in the form of a question, 
and in most cases the questions were directly related to the hypotheses and topic selected. Eleven 
(78.5%) of the PIPs included clearly stated study questions (Step 2.1). For some, the study purpose 
identified was not consistent with the remainder of the PIP (the target population, interventions, 


measures, or methods). 
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Table | - Performance Improvement Project Validation Findings by MC+ MCO. 


Selected Study Topics 





Step 2: Study Questions 


Step 3: Study Indicators 


Step 4: Study Populations 


Step 5: Sampling Methods 


Step 6: Data Collection Procedures 


Step 7: Improvement Strategies 


Step 8: Analysis and Interpretation of Study 
Results 


Step 9: Validity of Improvement 
Step 10: Sustained Improvement 
Number Met 

Number Partially Met 

Number Not Met 

Number Applicable 

Rate Met 


1.1 
1.2 
1.3 
Pad 
3.1 
3.2 
4.1 
4.2 
5.1 
5.2 
D3 
6.1 
6.2 
6.3 
6.4 
6.5 
6.6 
7.1 
8.1 
8.2 
8.3 
8.4 
9.1 
9.2 
9.3 
9.4 
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Improving Birthweight 
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Note: Rate Met = Number Met/Number Applicable; 2 = Met; | = Partially Met ; 0 = Not Met; NA = Not Applicable; Item refers to the Protocol specifications. 
Source: BHC, Inc., 2005 External Quality Review Performance Improvement Project Validation. 
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STEP 3: STUDY INDICATORS 

All PIPs either Met or Partially Met the criteria for defining and describing the calculation of study 
indicators. Eleven (78.5%) of the PIPs Met the criteria for using objective, clearly defined, 
measurable indicators while three were rated as Partially Met (CCP, MOCare, and FirstGuard PIPs) 
(Step 3.1). The calculation of measures was described and explained. Even when well-known 
measures were used (e.g., Health Employer Data Information Set; HEDIS; Consumer Assessment of 
Health Plans Survey; CAHPS), there was a description of the methods (e.g., Administrative or 
Hybrid Method) and formulas for calculating the measures. Again, because MC+ MCOs vary in their 
method of calculation, details regarding the measures and methods of calculating them should be 
included in PIPs. All but two of the I|4 PIPs identified at least one study indicator that was related to 
health or functional status; or to processes of care strongly associated with outcomes. Eight of the 
14 (57.1%) were rated as “Met” (Step 3.2). The link between the intervention and the outcomes 


measured by the PIP should be made explicit in the narrative. 


STEP 4: STUDY POPULATIONS 

Several of the PIPs failed to meet the criteria for clearly defining all the MC+ Managed Care Program 
Members to whom the study question(s) and indicator(s) were relevant. However, 9 of the |3 
(69.2%) did include adequate information to make this determination (Step 4.1). In two PIPS, which 
were considered non-clinical there was no applicable study population considered. The selection 
criteria should clearly describe the MC+ Managed Care Member populations included in the PIP and 
their demographic characteristics. Six of the |2 PIPs (50.0%) described data collection approaches 
indicating that data for all members to whom the study question applied were collected (Step 4.2). 
Some misunderstanding of sampling (e.g., “The sample size was determined by how many people we 
were able to contact during the quarter being measured’) continued from the 2004 EQR, but this 
only occurred in two of the narratives submitted. In most cases a description was included that 


allowed inference of how data were collected and how participants were identified. 


STEP 5: SAMPLING METHODS 
None of these PIPs employed true sampling techniques. The type of sample (e.g., convenience, 


random) or sampling methods (e.g., simple, cluster, stratified) should be described if utilized. 
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STEP 6: DATA COLLECTION PROCEDURES 

Twelve of the 14 PIPs (85.7%) described the data to be collected with adequate detail and 
description of the units of measurement used (Step 6.1). Eleven of the 14 (78.5%) PIPs clearly 
specified the sources of data (e.g., claims, members, providers, medical records) for each measure 
(Step 6.2). Some MC+ MCOs used the National Committee for Quality Assurance (NCQA) 
Quality Improvement Activity (QIA) Form to write PIPs, which provides a structure for reporting 
measures and data sources. However, when there is more than one source of data, it is important 
that the MC+ MCO specifically state the sources of data for each measure. It was noted to the MCt+ 
MCOs that the strict use of this format limits the narrative and explanation that must accompany 
the PIP to validate each element. Eight of the |4 PIPs (57.1%) clearly described systematic and 
reliable methods of data collection (Step 6.3). In one case, the data collection procedures were not 
described. It was not possible to judge the reliability or credibility of this PIP without sufficient 
detail regarding data collection processes, procedures, or frequency. Eleven of the PIPs used a 
survey or data collection tool, however, all were not presented in a method to be able to ensure 
that consistent and accurate data were collected over time (Step 6.4). However, five (45.5%) Met 
this element and five Partially Met this element. All but one of the MC+ MCOs that utilized surveys 
provided instruments for review. One PIP indicated that “the database” provided for accurate data 
collection. When using surveys, medical records, or telephone protocols for data collection, it is 
important to provide the tool for review, discuss the piloting of the tool, and discuss training and 
interrater reliability for the recording of information on the tool. Standard provider and consumer 
surveys provide manuals describing the characteristics of instruments that should be incorporated 
into the narrative of the PIP. This level of detail was not provided in the narrative, but in most cases 
the calculation of the measure did include sufficient information to make a judgment allowing 
determination of the rating for this validation element. Eight of the PIPs (57.1%) included a 
complete data analysis plan, while three additional PIPs were Partially Met for specifying a plan (Step 
6.5). This should be developed prior to the implementation of the PIP based on the study questions, 
expected relation between the intervention(s) and outcome(s) being measured (i.e. independent and 
dependent variables), the method(s) of data collection, and the nature of the data (e.g., nominal, 
ordinal, scale). Ten of the |4 (71.4%) PIPs identified in the narrative the project leader and 
qualifications of that individual, who was involved in or provided oversight for the design, 
implementation, data analysis, and interpretation of the PIP (Step 6.6). MC+ MCO staff interviewed 
on-site also included team members who were involved and knowledgeable about the PIPs and 


methods. Additional information about all the PIP team members and their qualifications and roles 
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were rarely described in detail, but would have provided additional clarification and validity to the 


process and the measures. 


STEP 7: IMPROVEMENT STRATEGIES 

Six of the 14 (42.9%) of the PIPs identified reasonable interventions to address the barriers identified 
through data analysis and quality improvement processes undertaken. Seven of the PIPs were 
Partially Met in this requirement. The nature of identification of the barriers, a description of 


barriers, and a plan for addressing barriers should be described. 


STEP 8: DATA ANALYSIS AND INTERPRETATION OF STUDY RESULTS 

Four of the 8 (50.0%) PIPs were not mature enough to have data to analyze. These MC+ MCOs 
conducted these analyses according to the data analysis plan (Step 8.1). Of the 8 PIPs that 
presented baseline or re-measurement data, two (25%) presented numerical findings accurately and 
clearly (Step 8.2). In some instances, data were presented in formats different from those described 
in the calculation of measures (e.g., presenting percents in graphic format while the description of 
the calculation of measures indicated rates per |,000). Six PIPs Partially Met this criteria. Axis labels 
and units of measurement should be reported in Tables and in Figure legends and this information 
should be made clearly identifiable to the reader. The data year for benchmark data should also be 
labeled. Of the six PIPs that presented at least one re-measurement period, four (66.6%) indicated 
the re-measurement period for all of the measures identified in the study (Step 8.3). Of the six PIPs 
describing the findings, three (50.0%) described the extent to which the intervention was effective 


(Step 8.4). 


STEP 9: VALIDITY OF IMPROVEMENT 

Four of the six PIPs (66.6%) with re-measurement points used the same method at re-measurement 
as the baseline measurement (Step 9.1). VWhenever possible the baseline measure should be 
recalculated consistent with the re-measurement method to ensure validity of reported 
improvement and comparability of measurement over time. One PIP did explain that the MC+ 
Medicaid eligibility criteria changed during the measurement year. How this change was 
incorporated into the baseline information was clearly explained and documented. The same source 
of measures should also be used at re-measurement points. Three of the PIPs (50.0%) that were 
mature enough to include data analysis employed statistical significance testing to document 


quantitative improvements in care (Step 9.2). They were able to show statistically significant 
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improvement over multiple re-measurement points. Three of five (60.0%) PIPs reporting 
improvements had face validity, meaning that the reported improvement was judged to have been 
related to the intervention applied (Step 9.3). These PIPs provided some discussion or 
interpretation of findings by MC+ MCOs. Additional narrative in this area would ensure proper 
evaluation of all data and information provided. After reporting findings, there should be some 
interpretation as to whether the intervention or other factors may have accounted for 
improvement, decline, or lack of change. Four of the five PIPs (80.0%) that had reached a level of 
maturity to include this data did provide statistical evidence that the observed improvement was 
true improvement (Step 9.4). Then, barriers should be identified and addressed for the next cycle 


of the PIP, or reasons for discontinuing the PIP should be described. 


STEP 10: SUSTAINED IMPROVEMENT 

Of the four PIPs examining multiple measurement points over time, only two (50.0%) PIPs used 
Statistical significance testing to demonstrate improvement. One PIP showed statistically significant 
improvement over several measurement points. The low numbers in this area are a function of the 


lack of maturity that many of the PIPs exhibited. 
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Table 2 - Summary of Performance Improvement Project Validation Ratings by Item, All MC+ MCOs 


All MC+ MCOs 
Number Number Number lotal Number 
Item Met Partially Met Not Met pVe)e)iver-jel(= Rate Met 


Step 1: Selected Study Topics aT 
S.C) |) 1) 
| =<) 


alicia aes 3 te) 78.57% 
3.2 gga) 57.20% 


Step 5: Sampling Methods 


Step 7: Improvement Strategies | 71,288 


Step 9: Validity of Improvement | S98.) #2) CCT 
92 —iSS~C~SCSSC*C 00H 
93 st~=<S~s~sSC<“<C*~‘“—s~sS*~‘C~*~SCS SCO, 
9 5 80,0056 





Number Met 165 79 13 257 64.20% 


Note: Percent Met = Number Met/ Number Applicable; Item refers to the Protocol specifications. 


Source: BHC, Inc., 2005 External Quality Review Performance Improvement Project Validation. 
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2.6 Conclusions 


Across all MC+ MCOs, the range in proportion of criteria that were Met for each PIP validated was 
[2.5% through 95.7%. Across all PIPs validated statewide, 64.2% of criteria were met. All sources 
of available data were used to develop the ratings for the PIP items. The EQRO comments were 
developed based on the written documentation and presentation of findings. All of the PIPs 
presented included thoughtful and complex information. For some of the PIPs, the MC+ MCO’s 
enhanced information obtained at the on-site review. This made it clear that the MC+ MCOs intend 
to use this process to improve organizational functions. In several cases the performance 
improvement project had already been incorporated into MC+ MCO daily operations. In all cases, 
there was enough information provided to validate the PIPs. On-site interviews and subsequent 
information provided revealed in-depth knowledge of the PIPs and detailed outcomes. PIPs are to 
be ongoing, with periodic re-measurement points. At least quarterly re-measurement is 
recommended to provide timely feedback to the MC+ MCO regarding the need to address barriers 
to implementation. MC+ MCO personnel involved in PIPs had extensive experience in clinical 
service delivery, quality improvement, and monitoring activities. It was clear that they had made a 
significant improvement and investment in designing valid evaluation studies using sound data 
collection and analysis methods. This requires technical expertise in health services research and/or 


program evaluation design. 


Based on the PIP validation process, at least five MC+ MCOs (Family Health Partners, FirstGuard, 
Blue-Advantage Plus of Kansas City, HealthCare USA, and Missouri Care) had active and ongoing 
PIPs as part of their quality improvement program. One MC+ MCO (Community CarePlus) 
significantly improved their utilization of the PIP process as a tool to develop their performance and 
improve services to members. This MC+ MCO’s PIPs were the least mature, but were vastly 
improved from prior submissions. The following summarizes the strengths, areas for improvement, 


and recommendations based on the findings of the Validation of Performance Improvement Projects 


activity. 
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Table 3 - Validity and Reliability of Performance Improvement Project Results 


PIP Name 

Emergency Room Utilization 

Early Intervention in Prenatal Care Management 
Medicaid Emergency Room Utilization 

Customer Service - Member Service Call Centers 
2005 Lead Performance Improvement Project 
2004/2005 Pre-Authorization I|mprovement Project 
Attention Deficit Hyperactivity Disorder 

Lead Performance Improvement Project 


Access to Primary Care Services 


Improving Lead Screening at 6-36 Months of Age 


Improving Birth Weight Outcomes 
Improving Asthma Medication Management 
Lead Testing |! mprovement Project 


Improving Care for Asthmatics 


Validation of PIPs 


Low Confidence 
Low Confidence 
Moderate Confidence 
Low Confidence 
Moderate Confidence 
High Confidence 
High Confidence 
Moderate Confidence 
Moderate Confidence 
High Confidence 
Low Confidence 
High Confidence 
Moderate Confidence 


High Confidence 





Note: Not Credible = There is little evidence that the study will or did produce results that could be attributed to the 
intervention(s); Low Confidence = Few aspects of the PIP were described or performed in a manner that would produce 
some confidence that findings could be attributed to the intervention(s); Moderate Confidence = Many aspects of the PIP 
were described or performed in a manner that would produce some confidence that findings could be attributed to the 
intervention(s); High Confidence = The PIP study was conducted or planned in a methodologically sound manner, with 
internal and external validity, standard measurement, and data collection practices, and appropriate analyses to calculate 
that there is a high level of confidence that improvements were a result of the intervention. A 95% to 99% level of 
confidence in the findings was or may be able to be demonstrated. 

Source: BHC, Inc., 2005 External Quality Review Performance Improvement Project Validation. 


STRENGTHS 


|. There was good topic identification and good intervention development for PIPs. MC+ MCO 
personnel were well qualified to identify study topics, areas in need of improvement, and 
interventions to address barriers to quality of care and health outcomes. 


2. MC+MCOs implemented interventions aimed at key aspects of enrollee care and services, such 
as medication and treatment management, risk identification and stratification for various levels 
of care, monitoring provider access, blood lead level screening, and preventive care. 


3. The descriptions of studies and intended purposes were often clear and relevant to the rationale 
and problem identified. 


4. Study indicators were well-defined and methods for calculating them were detailed. 


5. Study designs specified the data to be collected and the sources of data for each measure. 
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6. Quality improvement strategies were appropriate and relevant. 

7. The same methods and measures over re-measurement periods were used. 

8. Additional PIPS (Pre-Authorization Improvement, ADHD, Increased Blood Lead Level Testing at 
|2 & 24 Months, Access to Primary Care Services, Improving Lead Screening at 6-36 Months of 
Age, Improving Asthma Medication Management) were judged to be likely or highly likely to 
produce credible and valid findings to identify Best Practices. 

9. MC+ MCO personnel were attentive and responsive to technical assistance for the 
implementation of PIPs and presented goals or plans for improving the process in the future. 

10. MC+ MCOS instituted personnel and/or role changes to improve the focus and expertise of 


personnel responsible for PIP implementation. 


AREAS FOR IMPROVEMENT 





Study questions should continue to be refined. In some cases the study question did not frame 
the purpose, intended outcomes, goals, and methods for the study. 


There was one PIP that was focused almost exclusively on staff training and was not able to 
draw a relationship to improving services to members. PIPs should directly address the 
outcomes of care on health or functional status. 


Data analysis plans were not described in a number of the PIPs. The method of data collection 
and compilation is often dependent on the plan for data analysis. A plan should be developed 
prior to conducting a PIP, and modified as needed to be able to test for significant improvement 
or stability of performance on the outcome measures over time. Data analyses could not be 
evaluated without a description of the plan. 


There were PIPs underway or ongoing that resulted in the potential for credible findings. 
Ensuring that the project is started early enough to provide some data and data analysis is 
essential in completing the validation process. 


Although it appears that many MC+ MCOs conduct PIPs as part of their quality improvement 
program, the continued utilization of these PIPs as tools to improve the organizations ability to 
serve members would be beneficial. 
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RECOMMENDATIONS 





It is recommended that MC+ MCOs obtain additional training, assistance and expertise for the 
design, statistical analysis, and interpretation of PIP findings. One MC+ MCO (Children’s Mercy 
Family Health Partners) utilized the services of a statistician from a local university to ensure 
valid and reliable findings. 


In the design of PIPs, MC+ MCOs need to use generally accepted practices for program 
evaluation to conduct PIPs. In addition to training on the development of PIPs and on-site 
technical assistance, references to the CMS protocol, “Conducting Performance Improvement 
Projects” were recommended by the EQRO at each MC+ MCO as a guideline to frame the 
development, reporting and analysis of the PIP. 


PIPs should be conducted on an ongoing basis, with at least quarterly measurement of some 
indices to provide data about the need for changes in implementation, data collection, or 
interventions. 


It is recommended that a statewide PIP be identified by the SMA and the MC+ QA & | Group 
for planning and implementation one year prior to the planned implementation. The topic 
should be chosen based on the present study findings and current topic identification of MC+ 
MCOs as important processes that are likely to affect a broad segment of the population and 
key healthcare outcomes. 
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3.0 VALIDATION OF PERFORMANCE MEASURES 





3.1 Definition 


The Validating Performance Measures Protocol requires the EQRO to validate three performance 
measures at each MC+ MCO, as selected by the State Medicaid Agency (SMA; the Missouri 
Department of Social Services, Division of Medical Services; DMS). The three performance 
measures validated by the EQRO were the HEDIS 2005 measures of Well-Child Visits in the First 
I5 Months of Life, Childhood Immunization Status, Combination #2, and Annual Dental Visits. 
Protocol activities involved the review of the data management processes of the MC+ MCO, 
evaluation of algorithmic compliance with performance measure specifications, and verification of 
either the entire set or a sample of the performance measures to confirm that the reported results 


are based on accurate service information. 


3.2 Purpose and Objectives 


The objectives for validating performance measures were to: |) evaluate the accuracy of Medicaid 
performance measures reported by, or on behalf of, MC+ MCOs; and 2) determine the extent to 
which MC+ MCO-specific performance measures calculated by MC+ MCOs (or by entities acting on 
behalf of MC+ MCOs) followed specifications established by the SMA and the State Public Health 
Agency (SPHA; Missouri Department of Health and Senior Services; DHSS) for the calculation of the 


performance measure(s). 


REVIEWERS 

Two reviewers conducted the Validating Performance Measure Protocol activities, including 
interviews and document review. The External Quality Review Organization (EQRO) Project 
Director is a licensed attorney with a graduate degree in Health Care Administration, and five years 
of experience in public health and managed care in two states. She conducted interviews and 
document review. The EQRO Research Analyst is an Information Technology specialist with a 
Bachelors Degree in Information Systems and a Masters Degree in Business Administration. She has 
work for over two years managing data in large and small databases. She conducted interviews and 


data analysis. 
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3.3 Technical Methods 


The reliable and valid calculation of performance measures is necessary for calculating statewide 
rates; comparing MC+ MCO performance with other MC+ MCOs; and for comparing State and 
MC+ MCO performance with national benchmark data for Medicaid managed care and/or 
Commercial Managed Care Organization (MCO) members. These calculations allow MCO 
members to evaluate program effectiveness and access to care. State of Missouri requirements for 
MC+ MCO performance measurement and reporting were reviewed. The Missouri Code of State 
Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) contains provisions 
requiring all Health Maintenance Organizations (HMOs) operating in the State of Missouri to submit 
to the State SPHA member satisfaction survey findings and quality indicator data in formats 
conforming to the National Committee for Quality Assurance (NCQA) Health Employer Data 
Information Set (HEDIS) Data Submission Tool (DST) and all other HEDIS Technical Specifications? 
for performance measure descriptions and calculations. Additionally, the State of Missouri contract 
for Medicaid Managed Care (B3Z03182; Revised Attachment 6, Quality Improvement Strategy, 
08/13/2003) stipulates that MC+ MCOs will follow the instructions of the SPHA for submission of 
HEDIS measures. The three measures selected by the SMA for validation were required to be 
calculated and reported by MC+ MCOs to both the SMA and the SPHA for MC+ Managed Care 
Members. The HEDIS 2005 Technical Specifications were reviewed for each of the three measures 


and are summarized below (see Tables 4, 5, and 6). 


HEDIS 2005 CHILDHOOD IMMUNIZATION STATUS, COMBINATION #2 (CIS) 
The following is the definition of the Childhood Immunization Status measure, an Access to Care 


measure, as defined by the NCQA. 


The percentage of enrolled children two years of age who had four DTaP/DT, three IPV, one MMR, three H influenza type B, three 
hepatitis B and one chicken pox vaccine (VZV) by the time period specified and by their second birthday. The measure also 
calculates two separate combination rates. 


? National Committee for Quality Assurance (2003). HEDIS 2005, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 
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Table 4 - HEDIS 2005 Technical Specifications for Childhood Immunization Status, Combination #2 (CIS) 
Wm let iol(-mace) eleireiateyal 








Product line: Commercial, Medicaid (report each product line separately). 


Age: Children who turn two years of age during the measurement year. 


Sele Te Twelve months prior to the child’s second birthday. 

enrollment: 

No more than one gap in enrollment of up to 45 days during the 12 months prior to 

the child’s second birthday. To determine continuous enrollment for a Medicaid 

Allowable gap: beneficiary for whom enrollment is verified monthly, the member may not have more 
than a 1-month gap in coverage (i.e., a member whose coverage lapses for 2 months 
[60 days] is not continuously enrolled). 


Anchor date: Enrolled on the child’s second birthday. 
Benefit: Medical. 


Event/ diagnosis: None. 


Il. Administrative Specification 





Denominator: The eligible population. 


For all antigens, the MCO may count evidence of any of the following: 

7 Evidence of the antigen of combination vaccine, or 

7 Documented history of the illness, or 

» A seropositive test result. 
For combination vaccinations that require more than one antigen (ij.e., DTaP and 
MMR), the MCO must find evidence of all of the antigens. 


DTaP CPT: 90698, 90700, 90701, 90720, 90721, 90723; ICD-9-CM: 99.39 
Diptheria and tetanus CPT: 90702 

Diptheria CPT: 90719; ICD-9-CM: V02.4*, 032*, 99.36 

Tetanus CPT: 90703; I|CD-9-CM: 037*, 99.38 

Pertussis |CD-9-CM: 033*, 99.37 

OPV CPT: 90712 

IPV CPT: 90698, 90713, 90723; |CD-9-CM: V12.02*, 045*, 99.41 

MMR CPT: 90707, 90710; ICD-9-CM: 99.48 

Measles CPT: 90705, 90708; I|CD-9-CM: 055*, 99.45 

Mumps CPT: 90704,90709; ICD-9-CM: 072*, 99.46 

Rubella CPT: 90706,90708, 90709; I|CD-9-CM: 056*, 99.47 

HiB CPT: 90645,90646, 90647, 90648, 90698, 90720, 90721, 90748; |ICD-9-CM: 
041.5*, 038.41*, 320.0*, 482.2* 

Hepatitis B** CPT: 90723, 90740, 90744-90747, 90748; |CD-9-CM: V02.61*, 070.2*, 
070.3* 

VZV CPT: 90710, 90716; |CD-9-CM: 052*, 053* 


Numerators: 


*|ndicates evidence of disease. A member who has evidence of disease during the 
numerator event time is compliant for the antigen. 

** The two-dose hepatitis B antigen Recombivax is recommended for children 
between 11 and 14 years of age only. 
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Children who had a contraindication for a specific vaccine may be excluded from the 
denominator for all antigen rates and the combination rates. The denominator for all 
rates must be the same. An MCO that chooses to exclude contraindicated children 
may do so only for those children where the administrative data does not indicate that 
the contraindicated immunization was rendered. The exclusion must have occurred by 
the member’s second birthday. 


The MCO should look for contraindications as far back as possible in the member’s 
history and may use the following contraindications and codes* to identify allowable 
exclusions: 
Any particular vaccine - Contraindication: Anaphylactic reaction to the vaccine or its 
components; |CD-9-CM: 999.4 
DTaP - Contraindication: Encephalopathy; |CD-9-CM: 323.5 (must include E948.4 or 
Exclusion E948.5 or E948.6 to identify the vaccine) 
(optional): OPV, VZV and MMR - Contraindication: Immunodeficiency, including genetic 
(congenital) immunodeficiency syndromes; |CD-9-CM: 279 
OPV, VZV and MMR - Contraindication: HIV-infected or household contact with HIV 
infection; |CD-9-CM: Infection VO8, symptomatic 042 
OPV, VZV and MMR - Contraindication: Cancer of lymphoreticular or histiocytic tissue; 
|CD-9-CM: 200-202 
OPV, VZV and MMR - Contraindication: Multiple myeloma; |CD-9-CM: 203 
OPV, VZV and MMR - Contraindication: Leukemia; |CD-9-CM: 204-208 
OPV/IPV - Contraindication: Anaphylactic reaction to streptomycin, polymyxin B or 
neomycin 
HiB - Contraindication: None 
Hepatitis B - Contraindication: Anaphylactic reaction to common baker’s yeast 
VZV and MMR - Contraindication: Anaphylactic reaction to neomycin 


Ill. Hybrid Specification 





A systematic sample drawn from the eligible population for each product line. The 
MCO may reduce the sample size using the current year’s administrative rate for 


Denominator: combination 2 or the prior years audited, product-line specific results for combination 
2. For information on reducing sample size, refer to the Guidelines for Calculations 
and Sampling. 


For all antigens, the MCO may count any of the following: 

7 Evidence of the antigen or combination vaccine, or 

7 Documented history of the illness, or 

= A seropositive test result. 
For combination vaccinations that require more than one antigen (DTaP/DT and MMR), 
the MCO must find evidence of all of the antigens. 
Refer to the Administrative Specification above to identify positive numerator hits 
from the administrative data. 


Numerators: 


Administrative: 
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For immunization information obtained from the medical record, the MCO may count 
members where there is evidence that the antigen was rendered from: 
= A note indicating the name of the specific antigen and the date of the 
immunization, or 
=  Acertificate of immunization prepared by an authorized health care provider 
or agency including the specific dates and types of immunizations 
administered. 


For documented history of illness or a seropositive test result, the MCO must find a 

. note indicating the date of the event. The event must have occurred by the member’s 
Medical Record: 
second birthday. 

Notes in the medical record indicating that the member received the immunization “at 
delivery” or “in the hospital” may be counted toward the numerator. This applies only 
to immunizations that do not have minimum age restrictions (e.g., prior to 42 days 
after birth). 


A note that the “member is up to date” with all immunizations that does not list the 
dates of all immunizations and the names of the immunization agents does not 
constitute sufficient evidence of immunization for HEDIS reporting. 
Exclusion Refer to the Administrative Specification above for exclusion criteria. The exclusion 
(optional): must have occurred by the member’s second birthday. 


Source: MMWR January 16, 1988, Volume 47(01): 8-12 Note: NCQA follows the CDC/ACIP guidelines for 
immunizations. HEDIS implements these guidelines after three years to account for the look-back period in the measure and to allow 
the industry time to adapt to new guidelines. 
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An MCO that submits HEDIS data to NCQA must provide the following data elements: 


Table 5 - Data Elements for Childhood Immunization Status, Combination #2 


Administrative Hybrid 


Measurement year 

Data collection methodology (administrative or hybrid) 
Sampling method used 

Eligible population 


Number of numerator events by administrative data in eligible 
population (before exclusions) 


Current year’s administrative rate (before exclusions) 
Minimum required sample size (MRSS) or other sample size 
Oversampling rate 

Final sample size (FSS) 

Number of numerator events by administrative data in FSS 


Administrative rate on FSS 


Number of original sample records excluded because of valid data 
errors 

Number of records excluded because of contraindications identified 
through administrative data 

Number of records excluded because of contraindications identified 
through medical record review 


Number of employee/dependent medical records excluded 
Records added from the oversample list 

Denominator 

Numerator events by administrative data 

Numerator events by medical records 

Reported rate 

Lower 95% confidence interval 


Upper 95% confidence interval 


Source: National Committee for Quality Assurance (2005). HEDIS 2005, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 
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HEDIS 2005 WELL-CHILD VISITS IN THE FIRST 15 MONTHS OF LIFE (W15) 
The following is the general definition of the Well-Child Visit measure and the specific parameters as 


identified by NCQA. 


The percentage of enrolled members who turned |5 months old during the measurement year, who received either zero, 
one, two, three, four, five, six or more well-child visits with a primary care practitioner during their first |5 months of life. 


A child should be included in only one numerator (e.g., a child receiving six well-child visits will not be included in the rate 
for five, four or fewer visits). The MCO calculated seven rates for each of the two product lines (Medicaid and 
commercial). 


Table 6 - HEDIS 2005 Technical Specifications for Well-Child Visits in the First 15 Months of Life (W15) 
|. Eligible Population 
Product lines: Commercial, Medicaid (report each product line separately). 


Age: 15 months old during the measurement year. 
31 days - 15 months of age. Calculate 31 days of age by adding 31 days to 
the child’s date of birth. Calculated the 15-month birthday as the child’s first 
Continuous enrollment: birthday plus 90 days. For example, a child born on January 9, 2003, and 
included in the rate of six or more well-child visits must have had six well- 
child visits by April 9, 2004. 
No more than one gap in enrollment of up to 45 days during the continuous 
enrollment period. To determine continuous enrollment for a Medicaid 
Allowable gap: member for whom enrollment is verified monthly, the member may not have 
more than a 1-month gap in coverage (i.e., a member whose coverage 
lapses for 2 months [60 days] is not continuously enrolled). 


Anchor date: Enrolled on the day the child turns 15 months old . 
Benefit: Medical. 
Event/ diagnosis: None. 

Il. Administrative Specification 


Denominator: The eligible population. 


Seven separate numerators are calculated, corresponding to the number of 
members who received zero, one, two, three, four, five and six or more well- 
child visits with a primary care practitioner during their first 15 months of 
life. To count toward the measure, the well-child visit must occur with a 
primary care practitioner, but it does not have to be the practitioner 
assigned to the child. 


A child who had a claim/encounter from a primary care practitioner with a 

code listed below is considered to have received a well-child visit: 
Numerators: CPT Codes: 99381, 99382, 99391, 99432 

|CD-9-CM Codes: V20.2, V70.0, V70.3, V70.5, V70.6, V70.8, V70.9 


Note: An MCO with internal codes, or other transaction data not cited above for 
Medicaid members that denote an EPSDT well-child visit may use these codes as long as 
they document methods used to track EPSDT well-child visits. The MCO should refer to the 
Practitioner Turnover measure for the definition of primary care practitioner. The MCO may 
also count visits in primary care practitioner offices to physician assistants and nurse 
practitioners, even if the practitioner is not listed as a primary care practitioner in the MCO 
directory, as long as the practitioner has provided any of the specified services listed within 
the CPT and/or ICD-9-CM Codes listed above. 


Exclusion (optional): None. 
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Ill. Hybrid Specifications 
A systematic sample drawn from the MCO’s eligible population. 


The MCO may reduce its sample size using the current year’s administrative 

: ; rate for six or more visits, or the prior years audited, product line-specific 

Denominator: rate for six or more visits. 

Note: For information on reducing sample size, refer to the Guidelines for Calculations and 

Sampling. 

Seven separate numerators are calculated, corresponding to the number of 

members who received zero, one, two, three, four, five, six or more well- 

child visits with a primary care practitioner during their first 15 months of 

Numerators: life. To count toward this measure, the well-child visit must occur with a 
primary care practitioner. 


Refer to the Administrative Specification above to identify positive 
numerator hits from the administrative data. 
Documentation from the medical record must include a note indicating a visit 
with a primary care practitioner, the date the well-child visits occurred and 
evidence of all of the following: 

= A health and developmental history (physical and mental) 

= A physical exam 

7 Health education/anticipatory guidance. 


Administrative: 


Medical Record: 


= Preventive services may be rendered on the occasion of visits other than well-child 
visits. If the specified codes are present, these services count, regardless of the 
primary intent of the visit. 


= — The MCO may count services that occur over multiple visits toward this measure 
as long as all of the services occur within the time frame established in the 
measure. 


» —— Inpatient, emergency room and specialist visits do not count in this measure. The 
intent is to capture comprehensive well-child visits only. 


= — An MCO using the hybrid methodology may use a combination of administrative 
data and medical record review to identify well-child visits for an individual in the 
denominator as long as the dates of service are at least two weeks apart. For 
example, the MCO may count two well-child visits identified through 
administrative data and another visit identified through medical record review (a 
total of three well-child visits) for one member, if each visit shows a different date 
of service and the dates are at least two weeks apart. 


Exclusion (optional): 


= The MCO should refer to the Practitioner Turnover measure for the definition of 
primary care practitioner. This includes the use of non-physician practitioners 
such as nurse practitioners or physician assistant. 


s This measure is based on the CMS and American Academy of Pediatrics 

guidelines for EPSDT visits. The MCO should reference the American Academy of 
Pediatrics Guidelines for Health Supervision at www.aap.org and Bright Futures: 
Guidelines for Health Supervision of Infants, Children and Adolescents (published 
by the National Center for Education in Maternal and Child Health) at 
www.Brightfurutres.org for more detailed information of what constitutes a well- 
child visit. 

Source: National Committee for Quality Assurance (2005). HEDIS 2005, Volume 2: Technical Specifications. 

Washington, D.C.: NCQA. 
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An MCO that submits HEDIS data to NCQA must provide the following data elements: 


Table 7 - Data Elements for Well-Child Visits in the First 15 Months of Life 
Administrative Hybrid 


Measurement year 


Data collection methodology (administrative or hybrid) 


Sampling method used 
Eligible population 


Number of numerator events by administrative data in eligible 
population (before exclusions) 


Current year’s administrative rate (before exclusions) 
Minimum required sample size (MRSS) or other sample size 


Oversampling rate 


Final sample size (FSS) 


Number of numerator events by administrative data in FSS 


Administrative rate on FSS 


Number of original sample records excluded because of valid data 
errors 


Number of records excluded because of contraindications identified 
through administrative data 


Number of records excluded because of contraindications identified 
through medical record review 


Number of employee/dependent medical records excluded 
Records added from the oversample list 
Denominator 


Numerator events by administrative data Each of the 7 rates Each of the 7 rates 
Numerator events by medical records Each of the 7 rates 


Reported rate Each of the 7 rates Each of the 7 rates 


Lower 95% confidence interval x x 





Source: National Committee for Quality Assurance (2005). HEDIS 2005, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 
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HEDIS 2005 ANNUAL DENTAL VISIT (ADV) 


The following is the definition of the Annual Dental Visit measure, an Effectiveness of Care measure 


as defined by NCQA. 


The percentage of enrolled members 3-2! years of age who had at least one dental visit during the 
measurement year. This measure applies only if dental care is a covered benefit in the MCO’s Medicaid 
contract. 


Table 8 - HEDIS 2005 Technical Specifications for Annual Dental Visit (ADV) 
Wm altel iel(-m ace) oleit-iatela| 


Product lines: Medicaid. 


4-21 years as of December 31 of the measurement year. The measure is 
reported for each of the following age stratifications and as a combined rate: 
» 4-6 year olds 
7-10 year olds 
11-14 year olds 
15-18 year olds 
19-21 year olds 
Continuous 
The measurement year. 
enrollment: 
No more than one gap in enrollment of up to 45 days during the measurement 
year. To determine continuous enrollment for a Medicaid beneficiary for whom 
Allowable gap: enrollment is verified monthly, the member may not have more than a 1-month 
gap in coverage (i.e., a member whose coverage lapses for 2 months [60 days] is 
not considered continuously enrolled). 


Anchor date: Enrolled as of December 31 of the measurement year. 
Benefit: Dental. 
None. 


Event/ diagnosis: Note: Although the specifications are for ages 4-21 years of age, most 3-year-olds are included 
because the anchor date includes children whose 4*h birthday occurs anytime during the year up 
to December 31. 


I1. Administrative Specification 


Denominator: The eligible population for each age group and the combined total. 


One or more dental visits with a dental practitioner during the measurement 
year. A member had a dental visit if a submitted claim/encounter contains any of 
the codes listed below: 
CPT Codes: 70300, 70310, 70320, 70350, 70355 
|1CD-9-CM Codes: 23, 24, 87.11, 87.12, 89.31, 93.55, 96.54, 97.22, 97.33- 
97.35, 99.97 

pUMErAtor: HCPCS/ CDT-3 Codes: D0120-D0999, D1110-D1550, D2110-D2999, D3110- 
D3999, D4210-D4999, D5110-D5899, D6010-D6199, D7110-D7999, D8010- 
D8999, D9110-D9999 


Note: Current Dental Terminology (CDT) is the equivalent dental version of the CPT physician 
procedural coding system. 
Exclusion (optional): None. 


Ill. Hybrid Specification 


None. 
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An MCO that submits HEDIS data to NCQA must provide the following data elements: 


Table 9 - Data Elements for Annual Dental Visit 


Administrative 





Measurement year For each age stratification 
Data collection methodology (administrative) For each age stratification 
Eligible population For each age stratification 
Numerator events by administrative data For each age stratification 
Reported rate For each age stratification 
Lower 95% confidence interval For each age stratification 
Upper 95% confidence interval For each age stratification 


Source: National Committee for Quality Assurance (20054). HEDIS 2005, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 


METHODS OF CALCULATING PERFORMANCE MEASURES 

According to HEDIS technical specifications, there are two methods of calculating performance 
measures: |) the Administrative Method and 2) the Hybrid Method. The Well-Child Visits in the 
First 15 Months of Life and the Childhood Immunization Status measures permit the MCO to 
calculate the percentages (also referred to as rates) using either the Administrative Method or the 
Hybrid Method. The Annual Dental Visit measure is required to be calculated using only the 


Administrative Method. 


The Administrative Method involves examining claims and other databases (administrative data) to 
calculate the number of members in the entire eligible population who received a particular service 
(e.g., immunizations, well-child visits, or dental visits). The eligible population is defined by the 
HEDIS technical specifications. Those cases in which administrative data show that the member 
received the service(s) examined are considered “hits”, or “administrative hits.” The HEDIS 


technical specifications provide acceptable administrative codes for identifying an administrative hit. 


The Hybrid Method entails the selection of a random sample of members from the eligible 
population during the measurement year. For the Hybrid Method, administrative data are examined 
to select members eligible for the measure and to identify the number of members who received 
the service(s) as evidenced by a claim submission or through external sources of administrative data 
(e.g., State Public Health Agency Vital Statistics or Immunization Registry databases). Those cases in 
which there are no administrative data indicating that the member received the service or all of the 


services required to be an “administrative hit’ are identified for medical record review. 
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Documentation of all or some of the services in the medical record in combination with 


administrative data is considered a “hybrid hit.” 


Administrative hits and hybrid hits are summed to form the numerator of the rate of members 
receiving the service of interest (e.g., immunizations or well-child visit). The denominator of the 
rate is represented by the eligible population or those sampled from the eligible population. A 
simple formula of dividing the numerator into the denominator produces the rate reported to the 
SMA and the SPHA, expressed in percentages. There are a number of other specifications for 
sampling, oversampling, replacement, and treatment of contraindications for services that are further 
explained in the HEDIS 2005 Technical Specifications: Volume 2!° to which the interested reader is 


referred. 


TIME FRAME 

According to the HEDIS technical specifications, the time frame for including members in the eligible 
population or sample was the measurement year of calendar year (CY) 2004 for all the measures 
selected. The time frame for the events of interest (e.g., immunizations, well-child visits, and dental 


visits) was CY2003 and CY2004. 


PROCEDURES FOR DATA COLLECTION 

The HEDIS 2005 technical specifications for each measure validated were reviewed by the EQRO 
Project Director, Information Technology consultant, and a health management informatics 
consultant. Extensive training in data management and programming for healthcare quality indices, 
clinical training, research methods, and statistical analysis expertise were well represented among 
the personnel involved in adapting and implementing the Validating of Performance Measures 
Protocol to conform to the HEDIS, SMA, and SPHA requirements while maintaining consistency 
with the Validating Performance Measures Protocol. The following sections describe the 
procedures for each activity in the Validating Performance Measures Protocol as they were 


implemented for the three HEDIS 2005 measures validated. 


Pre-On-Site Activity One: Reviewer Worksheets 
Reviewer Worksheets were developed for the purpose of conducting activities and recording 
observations and comments for follow-up at the site visits. HEDIS 2005 technical specifications 


19 National Committee for Quality Assurance (2004). HEDIS 2005, Volume 2: Technical Specifications. 
Washington, D.C.: NCQA. 
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were used to refine the Reviewer Worksheets for the evaluation of each item. Throughout 
October and November 2005, project personnel met to review available source documents and 
develop the Reviewer Worksheets for conducting pre-on-site, on-site, and post-on-site activities as 
described below. The reviews formed the basis for completing the Attachments (V, VII, X, XII, XIll, 
and XV) of the Validating Performance Measures Protocol for each measure and MC+ MCO. 
Source documents used to develop the methods for review and complete the Attachments included: 


e Information Systems Capabilities Assessments (ISCA) developed by the SMA and completed by the MCOs 
during 2003, 2004 and 2005 

HEDIS 2005 Data Submission Tool (DST) 

HEDIS 2005 Baseline Assessment Tool (BAT) 

HEDIS 2005 Audit Report 

HEDIS 2005 SPHA Reports 


Pre-On-Site Activity Two: Preparation of MC+ MCOs 

Presentations, MC+ MCO orientation teleconference calls, and individual communications with 
personnel at MC+ MCOs responsible for HEDIS 2005 performance measure calculation were 
conducted between October 2005 and February 2006, with follow-up telephone calls and written 
communications continuing through May 2006. From October 21, 2005 through October 31, 2005, 
the EQRO conducted technical assistance orientation phone calls with each of the MC+ MCOs to 
provide education about the Validating of Performance Measures Protocol and the EQRO 
submission requirements. All written materials, letters and instructions were reviewed and 
approved by the SMA in advance. Technical objectives, methods, procedures, data sources, 
communication with the EQRO, and contact information for EQRO personnel were provided to 
MC+ MCOs prior to the teleconference calls. MC+ MCOs were requested to have in attendance 
the person(s) responsible for the calculation of the HEDIS 2005 performance measures validated. 
Teleconference meetings were led by the EQRO Project Director, with key project personnel and a 
representative from the SMA in attendance. Technical assistance was focused on describing the 
Validating Performance Measures Protocol; identifying the three measures validated; the purpose, 
activities and objectives of the EQRO; and defining the information and data needed for the EQRO 


to validate the performance measures. 


Additional technical assistance was provided to the attendees of the MC+ MCO Alll-Plan Meeting on 
October 20, 2005 by the EQRO Project Director and personnel. On November 7, 2005, formal 
written requests for data and information from the MC+ MCOs for the validation of performance 
measures were made by the EQRO, to be submitted by MC+ MCOs by December 9, 2005 (see 


Appendix 3). Detailed letters and instructions were mailed to QI/UM Coordinators and Medicaid 
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Plan Administrators explaining the type of information, purpose, and format of submissions. EQRO 
personnel were available and responded to electronic mail and telephone inquiries; and any 
requested clarifications throughout the evaluation process. The following are the data and 


documents requested from MC+ MCOs for the Validating Performance Measures Protocol: 


™ — HEDIS 2005 Data Submission Tool for all three measures for the MC+ Managed Care Population only. Do not 
include other measures or populations. 


=" 2005 HEDIS Audit Report. This is the HEDIS Performance Audit Report for the MC+ Managed Care Program 
product line and the three MC+ measures to be validated (complete report). If the three measures to be 
validated were not audited or if they were not audited for the MC+ Managed Care Program population, please 
send the report, as it contains Information Systems Capability Assessment Information that can be used as part 
of the Protocol. 


™ Baseline Assessment Tool for HEDIS 2005. The Baseline Assessment Tool is to include descriptions of the 
process for calculating measures for the MC+ Managed Care Program population. 


= List of cases for denominator with all HEDIS 2005 data elements specified in the measures. 


= List of cases for numerators with all HEDIS 2005 data elements specified in the measures, including fields for 
claims data and MOHSAIC, or other administrative data used. Please note that one of the review elements in 
the Protocol is: The “MCO/PIHP has retained copies of files or databases used for performance measure 
reporting, in the event that results need to be reproduced.” 


=" List of cases for which medical records were reviewed, with all HEDIS 2005 data elements specified in the 
measures. BHC will request MCOs to gather a maximum of 30 records per measure, based on a random sample 
and the MCO will be requested to send copies. 


=" — Sample medical record tools used for hybrid methods for the three HEDIS 2005 measures for the MC+ Managed 
Care Program population; and instructions for reviewers. 


=" — All worksheets, memos, minutes, documentation, policies and communications within the MCO and with HEDIS 
auditors regarding the calculation of the selected measures. 


™ — Policies, procedures, data and information used to produce numerators and denominators. 


=" Policies, procedures, and data used to implement sampling (if sampling was used). At a minimum, this should 
include documentation to facilitate evaluation of: 


e = Statistical testing of results and any corrections or adjustments made after processing. 

e Description of sampling techniques and documentation that assures the reviewer that samples used for 
baseline and repeat measurements of the performance measures were chosen using the same sampling 
frame and methodology. 

e Documentation of calculation for changes in performance from previous periods (if comparisons were 
made), including tests of statistical significance. 


Policies and procedures for mapping non-standard codes. 
™ — Record and file formats and descriptions for entry, intermediate, and repository files. 


™ Electronic transmission procedures documentation. (This will apply if MCO sends or receives data electronically 
from vendors performing the HEDIS abstractions, calculations or data entry.) 


Descriptive documentation for data entry, transfer, and manipulation programs and processes. 


=" — Samples of data from repository and transaction files to assess accuracy and completeness of the transfer 
process. 


Documentation of proper run controls and of staff review of report runs. 
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=" Documentation of results of statistical tests and any corrections or adjustments to data along with justification 
for such changes. 


=" Documentation of sources of any supporting external data or prior years’ data used in reporting. 


™ Procedures to identify, track, and link member enrollment by product line, product, geographic area, age, sex, 
member months, and member years. 


™ Procedures to track individual members through enrollment, disenrollment, and possible re-enrollment. 
™ Procedures used to link member months to member age. 


=" Documentation of “frozen” or archived files from which the samples were drawn, and if applicable, 
documentation of the MCO’s process to re-draw a sample or obtain necessary replacements. 


=" Procedures to capture data that may reside outside the MCO’s data sets (e.g. MOHSAIC). 


™ — Policies, procedures, and materials that evidence proper training, supervision, and adequate tools for medical 
record abstraction tasks. (May include training material, checks of inter-rater reliability, etc.) 


Pre-On-Site Activity Three: Assess the Integrity of the MCO's Information 
System 


The objective of this activity was to assess the integrity of the MC+ MCOs’ ability to link data from 
multiple sources. Once the Reviewer Worksheets were developed, EQRO personnel reviewed the 
prior year’s SMA-developed and administered Information Systems Capability Assessment (ISCA); 
the protocols require an ISCA to be administered every other year, therefore, six of the seven 
MCOs were not required to submit new information for the ISCA review this year. One MCO was 
required to submit new ISCA information due to their utilization of a different Information System 
following a buy-out. EQRO personnel also reviewed HEDIS 2005 Baseline Assessment Tool (BAT) 
submitted by each MC+ MCO. Detailed notes and follow-up questions were formulated for the site 


visit review. 


On-Site Activity One: Assess Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple sources 
and assess whether these abilities ensure the accuracy of the measures. The site visit activities 
addressed a series of technical, process, and competency reviews with MC+ MCO personnel 
(including management and technical staff) and vendors involved in the development and production 
of the HEDIS 2005 performance measures. The site visit activities examined the HEDIS 2005 
reporting processes, databases, software and vendors for the three measures validated. This 
included reviewing data processing issues for generating the rates and determining the numerator 
and denominator counts. Other activities involved generating ad-hoc reports based on similar 


criteria for at least one of the measures, reviewing database processing systems, software, 
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organizational reporting structures, and sampling methods. The following are the activities 
conducted at each MC+ MCO: 


e Review results of run queries (on-site observation, screen-shots, test output) 

e Examination of data fields for numerator & denominator calculation (examine field definitions and file 
content) 

e Review of applications, data formats, flowcharts, edit checks and file layouts 

e Review of source code, software certification reports 

e Review HEDIS repository procedures, software manuals 

e Test for code capture within system for measures (confirm principal & secondary codes, presence/absence 
of non-standard codes) 

e Review of operating reports 

e Review information system policies (data control, disaster recovery) 

e Review vendor associations & contracts 


The following are the interview questions developed for the site visits: 


e What are the processes of data integration and control within information systems? 

e What documentation processes are present for collection of data, steps taken and procedures to calculate 
the HEDIS measures? 

e What processes are used to produce denominators? 

e What processes are used to produce numerators? 

e How is sampling done for calculation of rates produced by the hybrid method? 

e How does the MCO submit the requirement performance reports to the State? 


From the site visit activities, interviews, and document reviews, Attachment V (Data Integration and 
Control Findings) of the Protocol was completed for each MC+ MCO and performance measure 


validated. 


On-Site Activity Two: Assess Documentation of Data and Processes Used to 
Calculate and Report Performance Measures 


The objectives of this activity were to assess the documentation of data collection, assess the 
process of integrating data into a performance measure set, and examine procedures used to query 


the data set to identify numerators, denominators, generate a sample, and apply proper algorithms. 


From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment VII (Data and Processes Used to Calculate and Report Performance) of the 
Protocol was completed for each MC+ MCO and measure validated. One limitation of this step 
was the inability of MC+ MCOs to provide documentation of processes used to calculate and report 
the performance measures due to the use of proprietary software or off-site vendor software and 


claims systems. 
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On-Site Activity Three: Assess Processes Used to Produce the Denominators 
The objectives of this activity were: |) to determine the extent to which all eligible members were 
included; 2) to evaluate programming logic and source codes relevant to each measure; and 3) to 


evaluate eligibility, enrollment, age, codes, and specifications related to each performance measure. 


The content and quality of the data files submitted were reviewed to facilitate the evaluation of 
compliance with the HEDIS 2005 technical specifications. Unlike the prior year’s review the MC+ 
MCOs did consistently submit the requested level of data (e.g., all elements required by the 
measures or information on hybrid or administrative data). In order to produce meaningful results, 
during the prior year’s audit, the EQRO allowed data to be submitted in any format that the MC+ 
MCOs could supply. However, for this year’s audit, the EQRO required that all the MC+ MCOs 
submit data in the format requested. This was a fairly successful approach, only one MC+ MCO did 
not supply the data in the format requested, more information regarding the incompleteness of that 


MC+ MCOQOs data is contained in the individual plan sections of this report. 


From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment X (Denominator Validation Findings) of the Protocol was completed for each 


MC+ MCO and performance measure validated. 


On-Site Activity Four: Assess Processes Used to Produce the Numerators 

The objectives of this activity were: |) to evaluate the MC+ MCOs’ ability to accurately identify 
medical events (e.g., immunizations, well-child visits, dental visits); 2) to evaluate the MCOs’ ability 
to identify events from other sources (e.g., medical records, State Public Immunization Registry); 3) 
to assess the use of codes for medical events; 4) evaluate procedures for non-duplication of event 
counting; 5) examine time parameters; 6) review the use of non-standard codes and maps; 7) 
identify medical record review procedures (Hybrid Method); and 8) to review the process of 


integrating administrative and medical record data. 


For the Administrative Method, validation of the numerators was conducted for all three measures 
using the specified parameters for the dates of service(s), diagnosis codes, and procedure codes as 

appropriate to the respective measures. For example, for all three measures, dates of service were 
required to occur between January |, 2004 and December 31, 2004. Cases with dates outside this 


range were considered not valid. 
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Validation of numerators for the Hybrid Method followed the Validating Performance Measures 
Protocol for sample selection and calculation of bias related to the medical record review. The 
Protocol requires the EQRO to sample up to 30 records from the medical records reported by the 
MC+ MCO as meeting the numerator criteria (hybrid hits). In the event that the MC+ MCO 
reported fewer than 30 numerator events from medical records, the EQRO requested all medical 
records that were reported by the MC+ MCO as meeting the numerator criteria. This approach 
does not apply to the Annual Dental Visits measure, as the Administrative Method of calculation is 


required for this measure by HEDIS technical specifications. 


Initial requests for documents and data were made on November 7, 2005, with submissions due to 
the EQRO by December 9, 2005. The EQRO required the MC+ MCOs to request medical records 
from the providers. On January 6, 2006, the MC+ MCOs were given a list of medical records to 
request, a letter from DMS explaining the purpose of the request, and the information necessary for 
the providers to send the medical records directly to the EQRO. The submission deadline for 
medical records was February 10, 2006. The record receipt rate was very good, of the 237 records 
contained in the sample, 235 were received by the EQRO for review. The timelines for medical 
record requests of a total of 60 or fewer records per MC+ MCO were within the five day time 
frame specified by the State of Missouri Medicaid Managed Care Contract for providing small 


numbers of medical records. 


The review of medical records was administered by Reliable Health Care, Inc. (RHC), a temporary 
healthcare services provider located in Kansas City, Missouri and a Business Associate of Behavioral 
Health Concepts, Inc., (the EQRO). RHC is a State of Missouri certified Minority-Owned Business 
Enterprise (MBE) operated by two registered nurses. RHC possesses expertise in recruiting nursing 
and professional health care staff for clinical, administrative, and HEDIS medical record review 
services. The review of medical records was conducted by RNs with over 20 years of clinical 
experience and who were currently licensed and practicing in the State of Missouri. Two RNs 
participated in the training and medical record review process and both had at least three years of 


experience conducting medical record reviews for HEDIS measures. 


Medical record abstraction tools for the Childhood Immunization Status, Combination #2, and the 
Well-Child Visits in the First 15 Months of Life measures were developed by the EQRO Project 
Director and revised in consultation with a nurse consultant, the EQRO Information Technology 


consultant and with the input of the nurse reviewers. The 2005 HEDIS technical specifications and 
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the Validating Performance Measures Protocol criteria were used to develop the medical record 
review tools and data analysis plan. A medical record review manual and documentation of ongoing 
reviewer questions and resolutions were developed for the review. A half day of training was 
conducted by the EQRO Project Director and staff on March 6, 2006 using sample medical record 
tools and reviewing all responses with feedback and discussion. The reviewer training and training 
manual covered content areas such as Health Insurance Portability and Accountability Act (HIPAA), 
confidentiality, conflict of interest, review tools, project background, Missouri’s Early and Periodic 
Screening, Diagnosis, and Treatment Program (EPSDT; the Healthy Children and Youth; HCY 
Program) and forms, Association for Professionals in Infection Control and Epidemiology (APIC) 
guidelines, and Bright Futures Guidelines (promulgated by the American Academy of Pediatricians). 
Teleconference meetings between the nurses, coders, and EQRO Project Director were conducted 
as needed to resolve questions and coding discrepancies throughout the duration of the medical 


record review process. 


A data entry format with validation parameters was developed for accurate medical record review 
data entry. A data entry manual and training were provided to the data entry person at RHC, Inc. 
Data was reviewed weekly for accuracy and completeness, with feedback and corrections made to 
the data entry person. The final databases were reviewed for validity, verified, and corrected prior 
to performing analyses. All data analyses were developed, reviewed, approved, and finalized by the 
EQRO Project Director. Attachments XII (Impact of Medical Record Findings) and XIll (Numerator 
Validation Findings) were completed based on the medical record review of documents and site visit 


interviews. 
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On-Site Activity Five: Assess Sampling Process (Hybrid Method) 
The objective of this activity was to assess the representativeness of the sample of care provided. 


e Review Information Systems Capability Assessment (ISCA) 

e Review HEDIS Baseline Assessment Tool (BAT) 

e Review Data Submission Tool (DST) 

e Review numerator and denominator files 

e Conduct medical record review for measures calculated using hybrid methodology 

e Determine the extent to which the record extract files are consistent with the data found in the medical 
records 

e Review of medical record abstraction tools and instructions 

e Conduct on-site interviews, activities, and review of additional documentation 

e For those MCQOs that calculated the Childhood Immunization and/or Well-Child Visits measures via hybrid 
methodology, a sample of medical records (up to 30) was conducted to validate the presence of 
immunizations that contributed to the numerator. 


From the review of documents and site visits, Attachment XV (Sampling Validation Findings) was 
completed for those MC+ MCOs that elected the Hybrid Method for the HEDIS 2005 Childhood 
Immunization Status, Combination #2 and the HEDIS 2005 Well-Child Visits in the First 15 Months 


of Life measure. 


On-Site Activity Six: Assess Submission of Required Performance Measures to 
State 


The objective of this activity was to assure proper submission of findings to the SMA and SPHA. 
The DST was obtained from the SPHA to determine the submission of the performance measures 
validated. Conversations with the SPHA representative responsible for compiling the measures for 
all MCOs in the State occurred with the EQRO Project Director to clarify questions, obtain data, 


and follow-up on MC+ MCO submission status. 


Post- On-Site Activity One: Determine Preliminary Validation Findings for 
each Measure 


Calculation of Bias 

The Validating Performance Measures Protocol specifies the method for calculating bias based on 
medical record review for the Hybrid Method. In addition to examining bias based on the medical 
record review and the Hybrid Method, the EQRO calculated bias related to the inappropriate 
inclusion of cases with administrative data that fell outside the parameters described in the HEDIS 
2005 technical specifications. For measures calculated using the Administrative Method, the EQRO 
examined the numerators and denominators for correct date ranges for dates of birth and dates of 
service as well as correct enrollment periods and codes used to identify the medical events. This 


was conducted as described above under on-site activities three and four. The estimated bias in the 
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calculation of the HEDIS 2005 measures for the Hybrid Method was calculated using the following 
procedures, methods and formulas, consistent with the Validating Performance Measures Protocol. 


Specific analytic procedures are described in the following section. 


Analysis 

Once the medical record review was complete, all administrative data provided by the MC+ MCOs 
in their data file submissions for the HEDIS 2005 Childhood Immunization Status, Combination #2 
measure were combined with the medical record review data collected by the EQRO. This allowed 
for calculation of the final rate by the EQRO for the multiple events (measles, mumps, rubella; MMR; 
and Hepatitis B vaccinations, Hi B, and VZV). The next step was to remove the duplicate 
immunizations documented in both the medical record and in administrative data. The number of 
vaccinations administered during the specified age range was counted for each record. Cases that 
met both criteria were counted as “hits’’ and considered valid numerators based on medical record 


review. 


For the HEDIS 2005 Well-Child Visits in the First 15 Months of Life measure, all administrative data 
provided by the MC+ MCOs in their data file submissions were combined with the medical record 
findings collected by the EQRO. This allowed for calculation of the final rate of multiple visits. In 
order for each event of a well-child visit to be met, there had to be documented evidence of all 
three components specified in the HEDIS 2005 technical specifications (a health and developmental 
history, physical exam, and health education/anticipatory guidance). The EQRO calculated the 
number of visits for every member of the eligible population, from zero visits to six or more visits. 


The rates for validation were matched with the MC+ MCOs reported rate for six or more visits. 


For the calculation of bias based on medical record review for the MC+ MCOs using the Hybrid 
Method for the HEDIS 2005 Childhood Immunization Status, Combination #2 and Well-Child Visits 
in the First 15 Months of Life measures, several steps were taken. First, the number of hits based 
on the medical record review was reported (Medical Records Validated by EQRO). Second, the 
Accuracy (proportion of Medical Records Validated by EQRO/Numerator Hits by Medical Records 
reported by the MCO) and Error Rates (100% - Accuracy Rate) were determined. Third, A Weight 
of Each Medical Record was calculated (100%/Denominator reported by the MCO) as specified by 
the Protocol. The number of False Positive Records was calculated (Error Rate * Numerator Hits 


by Medical Records reported by the MCO). This represents the number of records that were not 
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able to be validated by the EQRO. The Estimated Bias from Medical Records was calculated (False 
Positive Rate * Weight of Each Medical Record). 


To calculate the Total Estimated Bias in the calculation of the performance measures, the 
Administrative Hits Validated by the EQRO (through the previously described file validation 
process) and the Medical Record Hits Validated by the EQRO (as described above) were summed 
and divided by the Denominator reported by the MCO on the DST to determine the Rate Validated 
by the EQRO. The difference between the Rate Validated by the EQRO and the Rate Reported by 
the MC+ MCO to the SMA and SPHA was the Total Estimated Bias. A positive number reflects an 


overestimation of the rate, while a negative number reflects an underestimation of the rate. 


Once the EQRO concluded its on-site activities, the validation activity findings for each performance 
measure were aggregated. This involved the review and analysis of findings and Attachments 
produced for each performance measure selected for validation and for the MCO’s Information 
System as a result of pre-on-site and on-site activities. The EQRO Project Director reviewed and 
finalized all ratings before submitting to the SMA for final ratings on all Attachments, and completed 
the Final Performance Measure Validation Worksheets for all measures validated and MC+ MCOs. 
Ratings for each of the Worksheet items (0 = Not Met; | = Partially Met; 2 = Met) were summed 
for each worksheet and divided by the number of applicable items to form a rate for comparison to 
other MC+ MCOs. The worksheets for each measure were examined by the EQRO Project 


Director to complete the Final Audit Rating. 


Below is a summary of the final audit rating definitions specified in the Protocol. Any measures not 
reported were considered “Not Valid.” A Total Estimated Bias outside the 95% upper or lower 
confidence limits of the measures as reported by the MC+ MCO on the DST was considered not 


valid. 


Fully Compliant: Measure was fully compliant with State (SMA and SPHA) specifications. 


Substantially Compliant: Measure was substantially compliant with State (SMA and SPHA) 
specifications and had only minor deviations that did not significantly bias 
the reported rate. 


Not Valid: Measure deviated from State specifications such that the reported rate was 
significantly biased. This designation is also assigned to measures for which 
the data provided to the EQRO could not be independently validated. 
Significantly biased was defined by the EQRO as being outside the 95% 
confidence interval of the rate reported by the MC+ MCO on the HEDIS 2004 
Data Submission Tool. 
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Once the EQRO submitted its preliminary findings to the MC+ MCOs, including a summary 
Validation Worksheet for each performance measure validated (See Appendix 4), the MC+ MCOs 
were offered the opportunity to submit comments and documentation to support the correction of 
factual errors or omissions in the EQRO’s preliminary report (See Appendix 10). Several MC+ 


MCOs submitted corrective action plans to address issues identified in the preliminary report. 


3.4 Findings 


MC+ MCOs conduct the calculation of performance measures in collaboration with a variety of 
vendors and use a number of different management information systems to extract data for the 
calculation of measures. They are also required to undergo annual audits by NCQA-certified 
auditing firms that provide MC+ MCOs with recommendations for reporting or not reporting 
findings of specific measures to the NCQA. Regardless of the NCQA audit rating or rotation, MC+ 
MCOs are required to report the performance measures validated to the SMA and SPHA. Table 10 
summarizes the names of HEDIS-certified software used, medical record vendors, and HEDIS 
auditors. Tables || and 12 show the method of calculation used by each MC+ MCO and the audit 
ratings assigned by the NCQA Auditor. This information was taken from the NCQA-certified 
Auditors’ reports and MC+ MCO self-report to the EQRO. 


Table 10 - HEDIS 2005 Software, Vendors, and Auditors for the HEDIS 2005 Measures 


IE Vaat=meo) ma (tel (ere), Name of HEDIS 2005 
| MC+ MCO Name of Software Record Vendor PANU efi xelg 
Children’s Mercy Family Children’s Mercy Family 
Health Partners N/A Health Partners Qualis Health 

MS Access, MS Excel Healthcare Research 
Community CarePlus (Novasys) QMark/HEDISHelp Associates 

Health Plan Healthcare Research 
Mercy MC+ Reporter* Mercy Health Plan Associates 

Quality Spectrum* 


HEDIS repository by 
Catalyst Not Applicable. Do not 

HealthCare USA Technologies use Hybrid Method. HealthcareData.com, LLC 
Austin Provider 

Missouri Care Solutions Missouri Care Thomson MedStat 


FirstGuard Health Plan McKesson Primaris Ernst & Young, LLP 

Blue Advantage Plus of Software from ViPs, 

Kansas City Inc. MedMeasures QMark/HEDISHelp Ernst & Young, LLP 
Note: * NCQA-certified. 
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Table || - Summary of Method of Calculation Reported and Validated by MC+ MCOs 





Well-Child Visits in 
the first 15 Months fel alifelarexere| 
MC+ MCO of Life Immunization Status Annual Dental Visit 
Community CarePlus Administrative Hybrid Administrative 
Mercy MC+ Hybrid Hybrid Administrative 
HealthCare USA Administrative Administrative Administrative 


Missouri Care Hybrid Hybrid Administrative 
Children’s Mercy Family Health 
Partners Administrative Hybrid Administrative 
FirstGuard Health Plan Hybrid Hybrid Administrative 
Blue Advantage Plus of Kansas 
City Administrative Administrative Administrative 


Table 12 - Audit Designations from NCQA-Certified Auditors 
Well-Child Visits in fel avi Celareyere| 
the first 15 Immunization 
MC+ MCO i Months of Life Status Annual Dental Visit 
Community CarePlus Partial 
Mercy MC+ Full 
HealthCare USA Partial 


Missouri Care Partial 
Children’s Mercy Family 
Health Partners Partial 


FirstGuard Health Plan Partial 

Blue Advantage Plus of 

Kansas City Full 
Note: NA = Measure not audited; NR = Measure not reported; R = Measure reportable; NCQA = National Committee 
for Quality Assurance. 
Source: MCO self-report and NCQA Audit Report for HEDIS 2005 





The validation of each of the performance measures is discussed in the following sections with the 
findings from each validation activity described. Subsequent sections summarize the status of 
submission of the measures validated to the SMA and SPHA, the Final Audit Ratings, and 


conclusions. 


HEDIS 2005 CHILDHOOD IMMUNIZATION STATUS, COMBINATION #2 

Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple 
sources. It is based on the integrity of the management information systems and the ability to 
ensure accuracy of the measures. For the HEDIS 2005 Childhood Immunization Status, 
Combination #2 measure, the sources of data included enrollment, eligibility, claim files, and State 
Public Health Immunization Registry (MOHSAIC) data. Table |3 summarizes the findings of 
Attachment V (Data Integration and Control Findings) of the protocol. The rate of items that were 


met was calculated across MC+ MCOs and from the number of applicable items for each MC+ 


MCO. 
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Of all the MC+ MCOs that calculated the measure, 100% Met the criteria for having accurate and 
established procedures for transferring data into data repositories for calculation of the measure, 
coordinating data integration with vendors, and following standards associated with programming 
and testing. All criteria were met by six of the seven MC+ MCOs that calculated the measure 
(85.7%) except the one for retaining copies of files or databases used for performance measure 
reporting. One MC+ MCO (Mercy MC+) Partially Met the criteria for retaining copies of files or 
databases used for performance measure reporting due to a system change the MCO could not 
produce the database used for HEDIS 2005 calculations, they assured the EQRO that it was available 
on back-up tape. Each MC+ MCO Met 92.3% to 100.0% of the criteria for data integration and 


control. 
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Table |3 - Data Integration and Control Findings, HEDIS 2005 Childhood Immunization Status, Combination #2 Measure 


MC+ MCO All MC+ MCOs 





Number Number Not Number 
Audit Elements of of = HCUSA MOCare BA+ Partially Met Met PN eo) 0) ICeot-] ol (=) Rate Met 


MCO/PIHP processes accurately and completely transfer data 
from the transaction files (e.g., membership, provider, 
encounter/claims) into the repository used to keep the data 
until the calculations of the performance measures have been 


sae completed and validated. 2 2 2 2 2 2 2 7 O 0 7 100.0% 


52 Samples of data from repository are complete and accurate. 2 2 2 2 2 2 2 7 O 0 7 100.0% 


MCO’'s/PIHP’s processes to consolidate diversified files, and to 
extract required information from the performance measure 
5.3 repository are appropriate. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Actual results of file consolidations or extracts were consistent 
with those which should have resulted according to 
Sl documented algorithms or specifications. 2 2 2 2 2 2 2 7 0 re) 7 100.0% 


Procedures for coordinating the activities of multiple 
subcontractors ensure the accurate, timely, and complete 

5.5 integration of data into the performance measure database. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Computer program reports or documentation reflect vendor 


coordination activities, and no data necessary to performance 
measure reporting are lost or inappropriately modified during 


5.6 transfer. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The repository’s design, program flow charts, and source 
5.7 codes enable analyses and reports. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Proper linkage mechanisms have been employed to join data 
from all necessary sources (e.g., identifying a member with a 
5.8 given disease/condition). 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Examine and assess the adequacy of the documentation 
governing the production process, including MCO/PIHP 
production activity logs, and MCO/PIHP staff review of report 
5.9 runs. 2 2 2 2 2 2 2 7 0) 0 7 100.0% 


5.10 Prescribed data cutoff dates were followed. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


The MCO/PIHP has retained copies of files or databases used 
for performance measure reporting, in the event that results 
5.11 need to be reproduced. 2 1 2 2 2 2 2 6 1 0 7 85.7% 
Review documentation standards to determine the extent to 
which the reporting software program is properly documented 
with respect to every aspect of the performance measurement 
reporting repository, including building, maintaining, 
5.12 managing, testing, and report production. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Review the MCO’s/PIHP’s processes and documentation to 
determine the extent to which they comply with the MCO/PIHP 
standards associated with reporting program specifications, 


5.13 code review, and testing. 2 2 2 2 2 2 2 7 0 0 7 100.0% 

Number Met 13 12 13 13 13 13 13 Te) 1 fe) 91 98.9% 
Number Partially Met 0 1 0 0 0 0) 6) 
Number Not Met re) 0 O ) 6) 0) O 
Number Applicable 13 13 13 13 12 13 ule 
Rate Met 100.0% 92.3% 100.0% 100.0% 100.0% 100.0% 100.0% 

Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to 

the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially 


Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in 
documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Documentation of Data and Processes 

The objectives of this activity were to assess the documentation of data collection; the process of 
integrating data into a performance measure set; the procedures used to query the data set for 
sampling numerators and denominators; and the ability to apply proper algorithms. Table |4 
summarizes the findings of Attachment VI (Data and Processes Used to Calculate and Report 
Performance Measures) of the Protocol. Item 7.3 (Statistical testing of results and corrections made 
after processing) did not apply to the measure. Items 7.5, 7.7, and 7.10 were not applicable to 
HealthCare USA, as these items only apply to the use of the Hybrid Method of calculation. MC+ 
MCOs Met 82.8% of the criteria for applying appropriate data and process for the calculation of the 
HEDIS 2005 Childhood Immunization Status, Combination #2 measure. All MC+ MCOs (100.0%) 
Met the criteria for following data file and field definitions. Five of the seven MC+ MCOs used the 
Hybrid Method for calculation, and they all Met criteria for having detailed medical record review 
practices and reviewer training materials. External data sources (State Public Health Immunization 
Registry) for calculation of the measure were incorporated by six MC+ MCOs (85.7%) Community 
CarePlus did not incorporate State Public Health Immunization Registry (MOHSAIC) data into the 
calculation of the measure. Procedures for sampling were Met for all five MC+ MCOs that 
calculated the measure with the Hybrid method. Although MC+ MCOs frequently graphed the 
rates of performance over several years, only 42.9% used the calculation of statistical significance in 
rates from year to year as a measure of the significance of fluctuation in the measure (see items 7.8 
and 7.11). Each MC+ MCO calculating the measure Met 66.7% to 100% of the criteria for 


documentation of data and processes. 
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Table 14 - Data and Processes Used to Calculate and Report Performance Measures, HEDIS 2005Childhood Immunization Status, Combination #2 
MC+ MCO | All MC+ MCOs 












Number Number Not Number 
Item Audit Elements CccP MHP HCUSA MOCare FHP FG Number Met} Partially Met} Met Applicable | Rate Met 


Statistical testing of results and any corrections or 
7.3 adjustments made after processing. NA NA NA NA NA NA NA NA 


asta 100.0% 


eee eee 100.0% 


-,{/,],{[.,.],].,.].],][.].],_ 100.0% 
When determining improvement in performance between 

measurement periods, appropriate statistical methodology is 

applied to determine levels of significance of changes. 42.9% 


a a a7 82.5% 


Number Partially Met 

Number Not Met 

a a — fo 1. 7. 1 
Rate Met 66.7% 77.8% 83.3% 100.0% 77.8% 100.0% 66.7% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to 
the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially 
Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in 
documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 













Detailed medical record review methods and practices, 
including the qualifications of medical record review 
supervisor and staff; reviewer training materials; audit tools 
used, including completed copies of each record-level 
reviewer determination; all case-level critical performance 
measure data elements used to determine a positive or 
negative event or exclude a case from same; and inter-rater 
reliability testing procedures and results. 













If sampling used, description of sampling techniques, and 
documentation that assures the reviewer that samples used 
for baseline and repeat measurements of the performance 

measures were chosen using the same sampling frame and 
methodology. 









Data that are related from measure to measure are 
consistent (e.g., membership counts, provider totals, number 
of pregnancies and births). 
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Processes Used to Produce Denominators 

The objective of this activity was to determine the extent to which all eligible members were 
included in the denominator, evaluate the programming and logic source codes, and evaluate the 
specifications for calculating each measure. Table 15 summarizes the findings of Attachment X 
(Denominator Validation Findings) of the protocol. Items 10.5 (Identification of gender of the 
member), 10.6 (Calculation of member months or years), and 10.10 (Systems for estimating 
populations when they are unable to accurately be counted) were not applicable to this measure. 
Of the MC+ MCOs, 100% Met the criteria for producing denominators according to specifications. 
Six out of seven MC+ MCOs Met all the criteria for producing denominators according to 
specifications, one MC+ MCO (HealthCare USA) did not meet the requirement of “properly 
evaluating the completeness and accuracy of any codes used to identify medical events, such as 
diagnoses, procedures, or prescriptions, and these codes have been appropriately identified and 
applied as specified in each performance measure’ due to HealthCare USA’s not providing the 
codes to the EQRO for review. HealthCare USA cited the time it would take for them to provide 
the service codes as a reason for not providing them to the EQR. Each MC+ MCO calculating the 
measure Met the criteria for processes used to produce the denominators 85.7% tol00.0% of the 


time. 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 





Missouri MC+ Managed Care External Quality Review Section 3 





Report of Findings — 2005 Validation of Performance Measures 


Table 15 - Denominator Validation Findings, HEDIS 2005 Childhood Immunization Status, Combination #2 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements CCP HCUSA Le] @r- 1 =) BA+ Partially Met Met PV oye) Cer.) 0) (=) Rate Met 


All members who were eligible to receive the specified services 
were included in the initial population from which the final 
denominator was produced. This “at risk" population included 
both members who received the services, as well as those who 
did not. This same standard applies to provider groups or other 
relevant populations identified in the specifications of each 


10.1 performance measure. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


For each measure, programming logic or source code which 
identifies, tracks, and links member enrollment within and 
across product lines (e.g., Medicare and Medicaid), by age and 
sex, aS well as through possible periods of enrollment and 
disenrollment, has been appropriately applied according to the 
10.2 specifications of each performance measure. 2 D 2 2 > 2 2 7 0 0 7; 100.0% 





Calculations of continuous enrollment criteria were correctly 
10.3 carried out and applied to each measure (if applicable). P 3 2 5 > 5 g 7 0 0 7 100.0% 


Proper mathematical operations were used to determine patient 
10.4 age or range. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The MCO/PIHP can identify the variable(s) that define the 
member's sex in every file or algorithm needed to calculate the 
performance measure denominator, and the MCO/PIHP can 
explain what classification is carried out if neither of the required 


10.5 codes is present.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP has correctly calculated member months and 
110.6 member years, if applicable to the performance measure.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP has properly evaluated the completeness and 
accuracy of any codes used to identify medical events, such as 
diagnoses, procedures, or prescriptions, and these codes have 
been appropriately identified and applied as specified in each 
10.7 performance measure. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
Any time parameters required by the specifications of the 
performance measure are followed (e.g., cut off dates for data 
collection, counting 30 calendar days after discharge from a 


10.8 hospital, etc.). 2 2 2 2 2 2 2 7 0 0 Z. 100.0% 


Performance measure specifications or definitions that exclude 
members from a denominator were followed. For example, if a 
measure relates to receipt of a specific service, the denominator 
may need to be adjusted to reflect instances in which the patient 
10.9 refuses the service or the service is contraindicated. 2 ? 2 2 a 2 2 7 0 0 7 100.0% 


Systems or methods used by the MCO/PIHP to estimate 
populations when they cannot be accurately or completely 


10.10 counted (e.g., newborns) are valid.* NA NA NA NA NA NA NA 0 0 0 0 NA 
Number Met 7 7 6 7 7 7 7 48 ft) 1 Ks) 98.0% 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 0 1 0 0 0 0 
Number Applicable 7 7 7 7 7 7 7 
Rate Met 100.0% 100.0% 85.7% 100.0% 100.0% 100.0% 100.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates 
to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | 
=Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation 
in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Figure | illustrates the rate of eligible members per MC+ MCO, based on the enrollment of all MC+ 
MCO Waiver Recipients as of December 31, 2004 the end of the CY2004 measurement year. It was 
expected that MC+ MCOs would identify similar proportions of eligible members for the measure. The 
rate of eligible members (percent of eligible members divided by the total enrollment) was calculated for 
all MC+ MCOs. Two-tailed z-tests of each MC+ MCO comparing MC+ MCOs to the rate of eligible 
members for all MC+ MCOs were conducted at the 95% level of confidence. FirstGuard (4.10%) and 
HealthCare USA (3.93%) identified significantly higher rates of eligible members than the rate for all 
MC+ MCOs (3.64%), while Mercy Health Plan (2.78%) identified a significantly lower rate of eligible 
members. The difference in rates may be due to the demographic characteristics of the member 
population, the completeness of claims data, or the processes of identifying eligible members. The 
identification of eligible members for the HEDIS 2005 Childhood Immunization Status, Combination #2 
is dependent on the quality of the enrollment and eligibility files. 


Figure | - MC+ Managed Care Program HEDIS 2005 Childhood Immunization Status, Combination #2, Eligible 
Members 
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CGP MHP HCUSA MOCare FHP FG BA+ All MC+ MCOs 
@ 2005 Rate 3.11% 2.78% 3.93% 3.33% 3.78% 4.10% 3.54% 3.64% 
MC+ MCO 


Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. Enrollment as of the last week in December 2003 (the measurement year) was used to calculate 
the rate. 

Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); Missouri Department of Social Services, Division of 
Medical Services, State MPRI Session Screens, enrollment figures for all Waivers, December 31, 2003. 
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Processes Used to Produce Numerators 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to accurately identify medical 
events, evaluate the ability to identify events from other sources, evaluate procedures for non-duplicate 
counting of multiple events, review time parameters and the use of non-standard code maps, and assess 
the processes and procedures for collecting and incorporating medical record review data. Table 16 
shows the numerators, denominators, rates, and confidence intervals submitted by the MC+ MCOs to 
the SPHA on the DST for the HEDIS 2005 Childhood Immunization Status, Combination #2 measure. 
The rate for all MC+ MCOs was calculated by the EQRO, thus, there is no confidence interval reported 
for the statewide rate. The rate for all MC+ MCOs was 28.17%, with MC+ MCO rates ranging from 


18.46% (HealthCare USA) to 63.66% (Missouri Health Care). 


Table 16 - Data Submission for HEDIS 2005Childhood Immunization Status, Combination #2 


Final Data Administrative | Medical Record Total Hits 





Collection Denominator | Hits Reported by) Hits Reported Reported by Rate Reported 

MC+ MCO Method Used (DST) MCO (DST) by MCO (DST) MCO (DST) by MCO (DST) | LCL - UCL (DST) 
Blue Advantage Plus Administrative 1225 Doz NA 5O2 45.06% 42.23% - 47.89% 
Community Care Plus Hybrid 411 9 186 195 47.45% 42.50% - 52.39% 
Family Health Partners Hybrid 395 106 121 227 57.47% _ 52.47% - 62.47% 
FirstGuard Hybrid 411 128 101 229 55.72% 50.79% - 60.64% 
HealthCare USA Administrative 7258 1340 NA 1340 18.46% 14.03% - 15.89% 
Mercy Health Plan Hybrid 411 90 70 160 38.93% 34.09% - 43.77% 


Missouri Care Hybrid 410 63.66% 58.88% - 68.44% 


94 167 261 


Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral Health 
Concepts, Inc.); LCL = 95% Lower Confidence Limit; UCL = 95% Upper Confidence Limit. The denominator is either the eligible 
population (for administrative method calculation) or the sample size (for hybrid method calculation). The EQRO is charged with 
providing MCO to MCO comparisons. Therefore, the numerators and denominators for HealthCare USA were aggregated across all 
three regions, which were each 100% validated. Mercy Health Plans submitted data for Combination #2; Blue-Advantage Plus of Kansas 
City did not calculate the measure. The statewide rate for all MC+ MCOs was calculated by the EQRO using the sum of numerators 
divided by the sum of denominators. There was no statewide rate or confidence limits reported to the SMA or SPHA. 

Source: MC+ Managed Care Organization HEDIS 2005 Data Submission Tools (DST). 
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Figures 2, 3, and 4 illustrate the rates reported by the MC+ MCOs and the rates of administrative 
and hybrid hits for each MC+ MCO on the HEDIS 2005 Childhood Immunization Status, 
Combination #2 measure. The rate reported by each MC+ MCO was compared with the rate for 
all MC+ MCOs, with two-tailed z-tests conducted at the 95% confidence interval. The rate for all 
MC+ MCOs was lower than the National Commercial (72.5%) and the National Medicaid rates 
(62.9%). Children’s Mercy Family Health Partners, FirstGuard, and Missouri Care reported rates 
significantly higher than the rate for all MC+ MCOs (57.47%, 55.72% and 63.66%, respectively), while 
HealthCare USA's combined rate (18.46%) across all three regions was significantly below the rate 


for all MC+ MCOs (28.17%). 


Figure 2 - MC+ Managed Care Program HEDIS 2005 Childhood Immunization Status, Combination #2, 
Rates 
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All MC+ 
GP MHP HCUSA MOCare FHP FG BA+ 
MCOs 
™@ 2005 Rate 47.45% 38.93% 18.46% 63.66% 57.47% 55.72% 45.06% 28.17% 
MC+ MCO 


Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 

Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); National Committee for Quality Assurance 
(NCQA). 
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When the rates of administrative and hybrid hits were examined separately (see Figures 3 and 4), 
Blue-Advantage Plus of Kansas City and Children’s Mercy Family Health Partners reported 
significantly higher rates of administrative hits (45.06% and 30.63% respectively) than the rate for all 
MC+ MCOs (22.04%). However, Community CarePlus identified a significantly lower rate of 
administrative hits (2.19%) than the rate for all MC+ MCOs. This may be a function of the 
completeness of each MC+ MCOs claim system or claims for childhood immunizations. A possible 
reason for the low rate of administrative hits by Community CarePlus is the exclusion of the State 


Public Health Immunization Registry (MOHSAIC) data in calculating the numerators. 


Figure 3 - MC+ Managed Care Program HEDIS 2005 Childhood Immunization Status, Combination #2, 
Administrative Rate Only 
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m@ 2005 Rate 2.19% 21.90% 18.46% 22.93% 30.63% 24.57% 45.06% 22.04% 
MC+ MCO 


Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2005 Data Submission Tool (DST). 
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For hybrid hits, Community CarePlus and Missouri Care reported significantly higher rates of hybrid 
hits based on medical record review (45.26% and 40.73%, respectively) than the rate for all MC+ 
MCOs (21.76%) using the Hybrid Method. Mercy Health Plan reported a significantly lower rate of 
hybrid hits based on medical record review (17.03%) than the rate calculated across all MC+ MCOs. 
Differences may be due to the differences in processes for carrying out medical record reviews and 


compiling hybrid data to calculate the rate using the Hybrid Method. 


Figure 4 - MC+ Managed Care Program HEDIS 2005 Childhood Immunization Status, Combination #2, 
Hybrid Rate Only 
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@ 2005 Rate 45.26% 17.03% 40.73% 30.63% 24.57% 21.76% 
MC+ MCO 


Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 


significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2005 Data Submission Tool (DST). 


Tables |7 and 18 summarize the findings of the EQRO medical record review validation and 
Attachment XII (Impact of Medical Record Findings) of the Protocol. Five of the MC+ MCOs used 
the Hybrid Method of calculation. Three MC+ MCO selected a sample of 411 eligible members, 
Missouri Care selected a sample of 410 and Children’s Mercy Family Health Partners selected a 
sample of 395, these samples were consistent with HEDIS technical specifications. A total of 150 of 
the 645 medical records reported as hybrid hits by MC+ MCOs were sampled for validation by the 
EQRO. Only those records received were included in the validation. Of the 150 medical records 


sampled, 148 were received for review (98.67%), and 107 were able to be validated (72.30%), 
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resulting in an error rate of 28.7% across all MC+ MCOs using the Hybrid Method of calculation. 
The number of False Positive Records (the total amount that could not be validated) was 185 of the 
645 reported hits. The estimated bias for individual MC+ MCOs based on the medical record 
validation ranged from a 0.6% to 19.7% overestimate in the rate, with an estimated bias of 9.1% for 
all MC+ MCOs using the Hybrid Method. Table 19 shows the impact of the medical record review 
findings. The error rate ranged from 3.3% to 80.0%, with a rate of 28.7% for all MC+ MCOs using 
the Hybrid Method. The final estimated bias in the final rate ranged from -9.1 1% to 20.92%, with an 
average of 3.59% for all MC+ MCOs after taking into account the validation of administrative hits 


(see Item 12.8, Table 18). 


Table 19 shows the validation of numerators based on the review of numerator extract files and the 
medical record review. Item |3.6 was not applicable as none of the MC+ MCOs used non-standard 
codes to determine the numerators. Items |3.8 through 13.13 relate to the Hybrid Method and 
were not applicable to HealthCare USA. Across all MC+ MCOs, 95.8% of the criteria for calculating 
the numerator were met. Six of the seven (85.7%) MC+ MCOs calculating the measures Met 
criteria for using complete medical event codes, correctly classifying members for inclusion in the 
numerator, following time parameters for the specification of the measure, and capturing data for 
performance indicators that could be easily underreported due to services delivered outside the 
MC+ MCO. Community CarePlus did not meet the criteria due to the exclusion of MOHSAIC 
data. Six of the seven MC+ MCOs Met 91.7% to 100.0% of the criteria for processes used to 
produce the numerators. HealthCare USA Met 66.7% of the criteria for processes used to produce 
the numerators because they did not provide the EQRO with valid service codes to validate, 


therefore items |3.3 and 13.4 could not be Met. 
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Table 17 - Medical Record Validation ioe HEDIS ue cnilcnege Immunization Status, Combination #2 


Numerator | Number Medical | Number Medical | Number Medical] Rate 
Hits by | Records Sampled Records Records Validated of 
Denominator Medical for Audit by Received for Validated by LAX ={ote} ge [-) Accuracy WN Ce lah amey am =t-Lep 
Rate 


(Oem (0) (Sample Size) | Records (DST) =e] :X@) | Audit by EQRO =e] :{@) Received = Rate} Medical Record 


Soamunte Care Plus Community Care Pus AIL) 385302 69.0% 66.7% | 33.3% 0.002, 


ret a a 20.7% 20.0% | 80.0% 0,002 
sso ae 76.1% 76.7% 23.3% 0.002 


All MC+ MCOs PAI: } | 72.3% | 71.3% 28.7% 0.0005 
Note: DST = Data Submission Tool; EQRO = External ually Review Organization (Behavioral Health Concepts, re accuracy Rate = Number of Medical Records Validated by 

the EQRO/Number of Records Selected for Audit by EQRO; Error Rate = 100% - Accuracy Rate; Weight of Each Medical Record = Error Rate * Medical Record Hits Reported 

by MCO; Estimated Bias from Medical Records = Percent of bias due to the medical record review = False Positive Rate * Weight of Each Medical Record. 

Source: MC+ MCO Data Submission Tools (DST); BHC, Inc. 2005 External Quality Review Performance Measures Validation. 
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Table 18 - Impact of Medical Record Findings, HEDIS 2005Childhood Immunization Status, Combination #2 


: _MC+ MCO 
Item Audit Elements a a a HCUSA | MOCare ~~ FHP 


If no, the rest of the spreadsheet will be completed to determine the 
impact on the final rate. See Below NA NA See Below NA See Below NA 


Weight of Each Medical Record (Impact of each medical record on the 

final overall rate; determined by dividing 100% by the denominator) 0.002 0.002 0.002 
Expected Number of False Positives (Estimated number of medical 

records inappropriately counted as numerator positives) 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates 
to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation 
in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Table 19 - Numerator Validation Findings, HEDIS 2005Childhood Immunization Status, Combination #2 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements CCP HCUSA WM Lex er a=) BA+ Partially Met Met yAVoy eo) |(or-] 0) (=) Rate Met 


The MCO/PIHP has used the appropriate data, including 
linked data from separate data sets, to identify the entire at- 
134 risk population. 2 2 2 2 2 2 2 Z 0 0 7 100.0% 
The MCO/PIHP has in place and utilizes procedures to 
capture data for those performance indicators that could be 
easily under-reported due to the availability of services 


le outside the MCO/PIHP. 0 2 2 2 2 2 2 6 0 1 7 85.7% 





The MCO's/PIHP's use of codes used to identify medical 
events are complete, accurate, and specific in correctly 


133 describing what has transpired and when. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
when classifying members for inclusion or exclusion in the 

13.4 numerator. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
The MCO/PIHP has avoided or eliminated all double-counted 

13.5 members or numerator events. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Any non-standard codes used in determining thenumerator 
have been mapped to a standard coding scheme in a 
manner that is consistent, complete, and reproducible as 
evidenced by a review of the programming logic or a 
1306 demonstration of the program. NA NA NA NA NA NA NA 0 0 0 0 NA 
Any time parameters required by the specifications of the 
performance measure are adhered to (i.e., that the 
measured event occurred during the time period specified or 
13.7 defined in the performance measure). 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Meaical FECOrG FEVIEWS and aADSLUACLIONS Nave Deen Calrliea 
out in a manner that facilitates the collection of complete, 


13.8 accurate, and valid data. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
ReECOra TeEVIEW Slall Nave Deen Properly trainea ana 

13.9 supervised for the task. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
RECOIU AVSLIACLION! LOUIS FEQUITE LIC AGVPIOPIldale MOLALIOTI 

13.10 that the measured event occurred. D ») NA 2 2 1 NA 5 0 0 5 100.0% 
Record aDStraCuliOn LOOIS FeEquIre NOLALION OF LNe FESUILS OF 

13.11 findings of the measured event (if applicable). 2 2 NA 2 > 2 NA 5 0 0 5 100.0% 


Vadald INCiuaed If) LIC FECOIU CXLIACL IIES ale COPMSISLEML WILT) 
data found in the medical records as evidenced by a review 
of a sample of medical record for applicable performance 
measures. (From Medical Record Review Validation Tools- 
13.12 Table 5, ATTACHMENT XI!) 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
Ine Process OF INLEGFALING AQMINISLFAalLIve Gdala ana Medical 
record data for the purpose of determining the numerator is 


13.13 consistent and valid. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
Number Met if 12 4 12 12 12 6 ct) () 3 72 95.8% 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 1 0 2 0 0 0 0 
Number Applicable 12 12 6 12 12 12 6 
Rate Met 91.7% 100.0% 66.7% 100.0% 100.0% 100.0% 100.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Sampling Procedures for Hybrid Method 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to randomly sample from the 
eligible members for the measure when using the Hybrid Method of calculation. Table 20 
summarizes the findings of Attachment XV (Sampling Validation Findings) of the Protocol. Item 15.3 
(Each provider had an equal chance of being sampled) was not applicable to the HEDIS 2005 
Childhood Immunization Status, Combination #2 measure; and none of the items were applicable to 
HealthCare USA. Item 15.9 (Documenting if the requested sample size exceeded the eligible 
population size) did not apply to any of the MC+ MCOs for this measure. MC+ MCOs Met criteria 
(98.0%) for random sampling without systematic exclusion, examining files for bias, assuring there 
was no correlation between samples drawn, assuring members had the same chance of being 
included at baseline and follow-up measurement, maintaining sample files, meeting sample size 
requirements of the performance measure specifications, oversampling to accommodate for 
exclusions, and making substitutions properly. The criteria for sample exclusions was Met by four 
out of five (80.0%) of the MC+ MCOs. Mercy Health Plan systematically excluded medical records 
sampled for the Hybrid Method by not requesting all the records that did not have administrative 
hits for medical record review. Of the MC+ MCOs that calculated the measure, the rate for proper 


sampling ranged from 90.0% to 100.0%. 
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Table 20 - Sampling Validation Findings, HEDIS 2005Childhood Immunization Status, Combination #2 


1 [ O  (@L@) All MC+ MCOs 





Number Number Not Number 
Audit Elements BA+ Partially Met Met PVey el iet-]e) (=) Rate Met 


Each relevant member or provider had an equal chance of being 
15.1 selected; no one was systematically excluded from the sampling. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 


The MCO / PIHP followed the specifications set forth in the 
performance measure regarding the treatment of sample 
exclusions and replacements, and if any activity took place 
involving replacements of or exclusions from the sample, the 
15.2 MCO/PIHP kept adequate documentation of that activity. 2 0 NA 2 2 2 NA 4 0 1 5 80.0% 
Eacn provider serving a given number oT enrollees nad the same 
probability of being selected as any other provider serving the 
15.3 same number of enrollees.* NA NA NA NA NA NA NA 0 0 0 0 NA 
Ine MCU/PIHP examined Its sampled tiles Tor Dias, and IT any bias 
was detected, the MCO/PIHP is able to provide documentation that 
15.4 describes any efforts taken to correct it. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 


independently, and there is no correlation between drawn 


15.5 samples. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
Kelevant members or providers wno were not Included In tne 
sample for the baseline measurement had the same chance of 
being selected for the follow-up measurement as providers who 
11506 were included in the baseline. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
Ine MCU/PIHP Nas policies ana proceaures to maintain Tiles Trom 
which the samples are drawn in order to keep the population 
intact in the event that a sample must be re-drawn, or 
replacements made, and documentation that the original 


15.7 population is intact. 2 2 NA a 2 2 NA 5 0 0 5 100.0% 
sample sizes meet the requirements of the perrormance measure 
15.8 specifications. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 


The MCO/PIHP has appropriately handled the documentation and 
reporting of the measure if the requested sample size exceeds the 


15.9 population size. NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP properly oversampled in order to accommodate 
15.10 potential exclusions 2 2 NA 2 Z 2 NA 5 0 0 5 100.0% 


Substitution applied only to those members who met the exclusion 
criteria specified in the performance measure definitions or 





15.11 requirements. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
SuUDSTITUTIONS Were made Tor properly excluded recoras and tne 

15.12 percentage of substituted records was documented. 2 2 NA 2 2 2 NA 5 0 0 5 100.0% 
Number Met 10 9 0 10 10 10 0 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 1 0 0 0 0 0 
Number Applicable 10 10 0 10 10 10 0 
Rate Met 100.0% 90.0% NA 100.0% 100.0% 100.0% NA 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates 
to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in 
documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 


Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Submission of Measures to the State 

Reports from the SPHA were obtained regarding the submission of the HEDIS 2005 Childhood 
Immunization Status Combination #2 measure. All seven MC+ MCOs calculated and submitted the 
measure to the SPHA and SMA. All HMQs in the State of Missouri are required to calculate and 


report the measure to the SPHA, and MC+ MCOs are required to report the measure to the SMA. 


Final Validation Findings 

Table 21 shows the final data validation findings and the total estimated bias in calculation based on 
the validation of medical record data and review of the MC+ MCO extract files for calculating the 
HEDIS 2005 Childhood Immunization Status, Combination #2 measure. Figure 5 illustrates the 
differences between the rates reported to the SPHA and those calculated by the EQRO. The rate 
for all MC+ MCOs calculated based on data validated by the EQRO was 21.56%, while the rate 
reported by MC+ MCOs was 28.17% (see Table |6 and Figure 2), a 6.61% overestimate. 


Table 21 - Final Data Validation for HEDIS 2005Childhood Immunization Status, Combination #2 
Percentage of 
Medical 
Administrative Record Hits Total Hits Total 
Hits Validated by) Validated by Validated by Rate Reported | Validated by} Estimated 
Ob am (6 @) =e] <e) EQRO* =e] {e) by MCO (DST) 
Community Care Plus 66.67% 47.45% 32.36% 
Blue Advantage Plus NA 45.06% 27.98% 


Family Health Partners 96.67% 57.47% 57.21% 
FirstGuard 20.00% 55.72% 61.61% 


HealthCare USA (all 3 regions) NA 18.46% 15.22% 
Mercy Health Plan 96.67% 38.93% 38.36% 


Missouri Care 76.67% 63.66% 67.89% 
NT (ono (eel 58.78% 





Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral 
Health Concepts, Inc.); LCL = 95% Lower Confidence Limit; UCL = 95% Upper Confidence Limit; False Positive Records 
= Error Rate * Medical Record Hits Reported by MC+ MCO; Medical Record Hits Validated by the EQRO = Medical 
Record Hits Reported by MC+ MCO (DST) - False Positive Records; Total Estimated Bias = Rate Validated by EQRO 
using medical record review and data extract file review - Rate Reported by MC+ MCO (DST). Positive numbers 
represent an overestimate. The EQRO is charged with providing MCO to MCO comparisons. Therefore, the numerators 
and denominators for HealthCare USA were aggregated across all three regions, which were each 100% validated. Mercy 
Health Plans submitted data for Combination #2; Blue-Advantage Plus of Kansas City did not calculate the measure. 


*For a more detailed explanation of how Medical Record Hit percentages were calculated, please see Table | 7. 


Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Figure 5 - Rates Reported by MC+ MCOs and Validated by EQRO, HEDIS 2005 Childhood Immunization 
Status, Combination #2 
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MCOs 
MC+ MCO 


HM Rate Reported by MCO (DST) — 47.45% 45.06% 57.47% 55.72% 18.46% 38.93% 63.66% 28.17% 


O Rate Validated by EQRO 32.36% 27.98% 57.21% 61.61% 15.22% 38.36% 67.89% 21.56% 





Sources: MC+ MCO HEDIS 2004 Data Submission Tool (DST); BHC, Inc., 2005 External Quality Review Performance Measure 
Validation. *Rate calculated by EQRO is based on data provided to the EQRO for review; data provided could not be independently 
validated. 
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HEDIS 2005 WELL-CHILD VISITS IN THE FIRST 15 MONTHS OF LIFE 

Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple sources 
for the calculation of the HEDIS 2005 Well-Child Visits measure. It is related to the integrity of the 
management information systems and the ability to ensure accuracy of the measures. For the 
HEDIS 2005 Well-Child Visits measure, the sources of data included enrollment, eligibility, and claim 
files. Table 22 summarizes the findings of Attachment V (Data Integration and Control Findings) of 
the Protocol. The rate of items that were Met was calculated across MC+ MCOs and from the 


number of applicable items for each MC+ MCO. 


Across all MC+ MCOs, 98.9% of the criteria for data integration and control were Met. All MC+ 
MCOs (100.0%) Met the criteria for having accurate and established procedures for transferring 
data into data repositories for calculation of the measure, coordinating data integration with 


vendors, and following standards associated with programming and testing. 
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Table 22 - Data Integration and Control Findings, HEDIS 2005 Well-Child Visits in the First 15 Months of Life 


MC+ MCO All MC+ MCOs 





Number Number Not Number 
Audit Elements (ee HCUSA MOCare BA+ Partially Met Met yAVoy o) Cet} 0} (=) Rate Met 


MCO/PIHP processes accurately and completely transfer data 
from the transaction files (e.g., membership, provider, 
encounter/claims) into the repository used to keep the data 
until the calculations of the performance measures have been 


5.1 completed and validated. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Bee Samples of data from repository are complete and accurate. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


MCO’s/PIHP’s processes to consolidate diversified files, and to 
extract required information from the performance measure 
5.3 repository are appropriate. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Actual results of tile consolidations or extracts were consistent 
with those which should have resulted according to 
5.4 documented algorithms or specifications. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Procedures for coordinating the activities of multiple 
subcontractors ensure the accurate, timely, and complete 

5.5 integration of data into the performance measure database. 2 , 2 2 2 2 2 Fr, 0 0 7 100.0% 
Computer program reports or documentation reflect vendor 


coordination activities, and no data necessary to performance 
measure reporting are lost or inappropriately modified during 


ye) transfer. 2) 2 2) Z 2 Z Z 7 0 0 7 100.0% 
The repository’s design, program flow charts, and source 
ae | codes enable analyses and reports. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Proper linkage mechanisms have been employed to join data 
from all necessary sources (e.g., identifying a member with a 
5.8 given disease/ condition). 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Examine and assess the adequacy of the documentation 
governing the production process, including MCO/PIHP 
production activity logs, and MCO/PIHP staff review of report 
5.9 runs. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


5.10 Prescribed data cutoff dates were followed. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


The MCO/PIHP has retained copies of files or databases used 
for performance measure reporting, in the event that results 
5.11 need to be reproduced. 2 1 2 2 2 2 2 6 1 0 7 85.7% 
Review documentation standards to determine the extent to 
which the reporting software program is properly documented 
with respect to every aspect of the performance measurement 
reporting repository, including building, maintaining, 
5.12 managing, testing, and report production. 2 2 2 2 2 2 2 Fi 0 0 7 100.0% 


Review the MCO’s/PIHP’s processes and documentation to 
determine the extent to which they comply with the MCO/PI HP 
standards associated with reporting program specifications, 


5.13 code review, and testing. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Number Met 13 12 13 13 13 13 13 90 1 (0) 91 98.9% 
Number Partially Met 0 1 0 0 0 0 0 
Number Not Met 0 ) ) 0 0 0 0 
Number Applicable 13 13 13 13 13 13 13 
Rate Met 100.0% 92.3% 100.0% 100.0% 100.0% 100.0% 100.0% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates 
to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in 
documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Documentation of Data and Processes 

The objectives of this activity were to assess the documentation of data collection; the process of 
integrating data into a performance measure set; the procedures used to query the data set for 
sampling, numerators and denominators; and the ability to apply proper algorithms for the calculation of 
HEDIS 2004 Adolescent Well-Care Visits measure. Table 23 summarizes the findings of Attachment VI 
(Data and Processes Used to Calculate and Report Performance Measures) of the Protocol. Items 7.3 
(Statistical testing of results and corrections made after processing), 7.4 (Inclusion of external data 
sources), and 7.9 (Consistent data from measure to measure) did not apply to the measure. Across all 
MC+ MCOs, 79.5% of the criteria were met. All MC+ MCOs Met the criteria for following data file and 
field definitions (100.0%). Three of the seven MC+ MCOs (42.9%) used the Hybrid Method for 
calculation, and all three Met criteria for having detailed medical record review practices and reviewer 
training materials. Source codes and programming logic for the identification of denominators appeared 
accurate for all six of the seven MC+ MCOs (85.7%). One MC+ MCO (HealthCare USA) did not 
provide source code or programming logic to the EQRO for review. All three (100.0%) MC+ MCOs 
using the Hybrid Method Met the criteria for documentation of sampling procedures. Although MC+ 
MCOs frequently graphed the rates of performance over several years, only 42.9% used the calculation 
of statistical significance in rates from year to year as a measure of the significance of fluctuation in the 
measure (see items 7.8 and 7.| 1). When sampling, all three (100.0%) of the MC+ MCOs using the 
Hybrid Method Met the criteria for using appropriate statistical functions for determining confidence 
intervals for sampling. Each MC+ MCO calculating the measure Met 60.0% to 100.0% of the criteria for 
processes used to calculate and report the HEDIS 2005 Well-Child Visits in the First 15 Months of Life 


measure. 
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Table 23 - Data and Processes Used to Calculate and Report Performance Measures, HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 


Mc+ MCO | All MC+ MCOs 







Number Number Not Number 


Item Audit Elements HCUSA MOCare _ Number Met} Partially Met Met | PAV oye) Cet-} o) (=) | Rate Met 


Statistical testing of results and any corrections or 
7.3 adjustments made after processing. NA NA NA NA NA NA NA 


Detailed medical record review methods and practices, 
NA 2 NA 2 NA 2 NA 3 3 


including the qualifications of medical record review supervisor 
_ _ 


and staff; reviewer training materials; audit tools used, 
including completed copies of each record-level reviewer 

Pa i 
when determining Improvement in perrormance between 
measurement periods, appropriate statistical methodology is 
applied to determine levels of significance of changes. 


determination; all case-level critical performance measure 


data elements used to determine a positive or negative event 
or exclude a case from same; and inter-rater reliability 

testing procedures and results. 

Number Partially Met 0 

Number Not Met 2 2 1 2 0 

WumiberAppieable 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 






100.0% 





































If sampling used, description of sampling techniques, and 
documentation that assures the reviewer that samples used 
for baseline and repeat measurements of the performance 

measures were chosen using the same sampling frame and 
methodology. 
















Data that are related from measure to measure are consistent 
(e.g., membership counts, provider totals, number of 
pregnancies and births). 
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Processes Used to Produce Denominators 

The objective of this activity was to determine the extent to which all eligible members were 
included in the denominator, evaluate the programming and logic source codes, and evaluate the 
specifications for each measure. For the HEDIS 2005 Well-Child Visits measure, the sources of data 
include enrollment, eligibility, and claim files. Table 24 summarizes the findings of Attachment X 
(Denominator Validation Findings) of the protocol. Items 10.5 (Identification of gender of the 
member), 10.6 (Calculation of member months or years), and 10.10 (Systems for estimating 
populations when they are unable to accurately be counted) were not applicable to the HEDIS 2005 
Well-Child Visits in the First 15 Months of Life measure. Six of the seven MC+ MCOs (85.7%) Met 
all the criteria for processes used to produce the denominators. One MC+ MCO (HealthCare 
USA) did Not Met the criteria for “evaluating the completeness and accuracy of any codes used to 
identify medical events, such as diagnoses, procedures, or prescriptions, and these codes have been 
appropriately identified and applied as specified in each performance measure’, as they did not 
provide service codes to the EQRO for review. 97.6% of the criteria were Met for the processes 


used to produce denominators. 
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Table 24 - Denominator Validation Findings, HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements CCP HCUSA MOCare BA+ Partially Met Met Applicable Rate Met 


All members who were eligible to receive the specified services 
were included in the initial population from which the final 
denominator was produced. This "at risk" population included 
both members who received the services, as well as those who 
did not. This same standard applies to provider groups or other 
relevant populations identified in the specifications of each 


10.1 performance measure. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


For each measure, programming logic or source code which 
identifies, tracks, and links member enrollment within and 
across product lines (e.g., Medicare and Medicaid), by age and 
sex, aS well as through possible periods of enrollment and 
disenrollment, has been appropriately applied according to the 
10.2 specifications of each performance measure. 2 2 2 2 2 2 > 7 0 0 7 100.0% 





Calculations of continuous enrollment criteria were correctly 
10.3 carried out and applied to each measure (if applicable). 2 o ri 2 PI 2 2 4 0 0 7 100.0% 


Proper mathematical operations were used to determine patient 
10.4 age or range. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The MCO/PIHP can identify the variable(s) that define the 
member's sex in every file or algorithm needed to calculate the 
performance measure denominator, and the MCO/PIHP can 
explain what classification is carried out if neither of the required 


10.5 codes is present.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP has correctly calculated member months and 
110.6 member years, if applicable to the performance measure.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP has properly evaluated the completeness and 
accuracy of any codes used to identify medical events, such as 
diagnoses, procedures, or prescriptions, and these codes have 
been appropriately identified and applied as specified in each 
10.7 performance measure. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
Any time parameters required by the specifications of the 
performance measure are followed (e.g., cut off dates for data 
collection, counting 30 calendar days after discharge from a 
10.8 hospital, etc.). 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Performance measure specifications or definitions that exclude 
members from a denominator were followed. For example, if a 
measure relates to receipt of a specific service, the denominator 
may need to be adjusted to reflect instances in which the patient 
10.9 refuses the service or the service is contraindicated. NA NA NA NA NA NA NA 0 0 0 0 NA 


Systems or methods used by the MCO/PIHP to estimate 
populations when they cannot be accurately or completely 
10.10 counted (e.g., newborns) are valid.* NA NA NA NA NA NA NA 0 0 0 0 NA 


Number Met 


6 6 5 6 6 6 6 41 (0) 1 42 ey 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 0 1 0 0 0 0 
Number Applicable 6 6 6 6 6 6 6 
Rate Met 100.0% 100.0% 83.3% 100.0% 100.0% 100.0% 100.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Figure 6 illustrates the rate of eligible members identified by each MC+ MCO, based on the 
enrollment of all MC+ MCO Waiver Recipients as of December 31, 2004, the end of the CY2004 
measurement year. It was expected that MC+ MCOs would identify similar proportions of eligible 
members for the HEDIS 2005 Well-Child Visits in the First 15 Months of Life measure. The rate of 
eligible members (percent of eligible members divided by the total enrollment) was calculated for all 
MC+ MCOs and two-tailed z-tests of each MC+ MCO compared to the state rate of eligible 


members were conducted at the 95% level of confidence. 


Figure 6 - MC+ Managed Care Program HEDIS 2005 Well-Child Visits in the First 15 Months of Life, 
Eligible Members 
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All MC+ 
CGP MHP HCUSA MOCare FHP FG BA+ 
MCOs 
@ 2005 Rate 2.83% 2.71% 3.50% 3.26% 3.27% 3.73% 3.47% 3.60% 
MC+ MCO 


Note: Error bars on the y-axis represent 95% confidence intervals; there were no significant differences on two tailed z- 
tests. 

Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); Missouri Department of Social Services, Division 
of Medical Services, State MPRI Session Screens, enrollment figures for all Waivers, December 31, 2004. 
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Processes Used to Produce Numerators 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to accurately identify medical 
events, evaluate the ability to identify events from other sources, evaluate procedures for non- 
duplicate counting of multiple events, review time parameters and the use of non-standard code 
maps, and assess the processes and procedures for collecting and incorporating medical record 
review data. For the HEDIS 2005 Well-Child Visits measure, the sources of data included 
enrollment, eligibility, and claim files. Table 25 shows the numerators, denominators, rates, and 
confidence intervals submitted by the MC+ MCOs to the SPHA on the DST. The rate for all MC+ 
MCOs was calculated by the EQRO, thus there is no confidence interval reported for the statewide 
rate. The rate for all MC+ MCOs was 38.42%, with MC+ MCO rates ranging from 23.91% 
(Children’s Mercy Family Health Partners) to 75.69% (Missouri Care). 
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Table 25 - Data Submission for HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 
Final Data Administrative Hybrid Hits 
Collection Denominator | Hits Reported by Reported by 

(Or am vy [ ©) Method Used (DST) MCO (DST) MCO (DST) 


i, J 


Blue Advantage Plus Administrative 1,201 


Total Hits 
Reported by Rate Reported 
MCO (DST) by MCO (DST) |LCL- UCL (DST) 


Family Health Partners Administrative 1,677 23.91% 21.84% - 25.98% 


HealthCare USA 6,474 2,687 2,687 41.50% 37.33% - 40.03% 


All MC+ MCOs 11,767 4,390 | 131 4,521 38.42% 





Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); LCL = 95% Lower 
Confidence Limit; UCL = 95% Upper Confidence Limit. The EQRO is charged with providing MCO to MCO comparisons. Therefore, the numerators and denominators 
for HealthCare USA were aggregated across all three regions, which were each 100% validated. Medical Record Hits Validated by EQRO = Weighted number of medical 
records with validated hits. Community CarePlus reported numerator events from Calendar Year 2002 for the HEDIS 2004 reporting year. The statewide rate for all MC+ 
MCOs was calculated by the EQRO using the sum of numerators divided by sum of denominators. There was no statewide rate or confidence limits reported to the SMA 
or SPHA. 

Source: MC+ Managed Care Organization HEDIS 2005 Data Submission Tools (DST). 
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Figures 7, 8, and 9 illustrate the rates reported by the MC+ MCOs and the rates of administrative 
and hybrid hits for each MC+ MCO. The rate reported by each MC+ MCO was compared with the 
rate for all MC+ MCOs. Two-tailed z-tests of each MC+ MCO comparing MC+ MCOQOs to the rate 
for all MC+ MCOs were calculated at the 95% confidence interval. The rate for all MC+ MCOs was 
lower than the National Commercial (68.70%) and the National Medicaid rates (45.00%). Missouri 
Care and First Guard reported rates significantly higher than the rate for all MC+ MCOs (75.69% 
and 51.58% respectively), while Children’s Mercy Family Health Partners reported a rate significantly 
below the rate for all MC+ MCOs (23.91%). 


Figure 7 - MC+ Managed Care Program HEDIS 2005 Well-Child Visits in the First 15 Months of Life, Rates 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 

Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); National Committee for Quality Assurance 
(NCQA). 
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When the rate of administrative and hybrid hits was examined separately, there was wide variability 
among MC+ MCOs from the administrative rate for all MC+ MCOs (37.31%). HealthCare USA and 
Missouri Care reported rates of administrative hits significantly higher than the rate for all MC+ 
MCOs (41.50% and 67.36%, respectively). Community CarePlus, Childrens Mercy Family Health 
Partners, Blue-Advantage Plus of Kansas City and Mercy Health Plan reported significantly lower 
rates of administrative hits (32.64%, 23.91%, 33.06% and 32.36%, respectively), while the rate found 
by FirstGuard was consistent with that of the rate for all MC+ MCOs. This may be a function of the 


completeness of each MC+ MCOs’ claim system or the administration of claims for well-child visits. 


Figure 8 - MC+ Managed Care Program HEDIS 2005 Well-Child Visits in the First 15 Months of Life, 
Administrative Rate Only 
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@ 2005 Rate 32.64% 32.36% 41.50% 67.36% 23.91% 36.98% 33.06% 37.31% 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2005 Data Submission Tool (DST). 
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For hybrid hits, First Guard reported a significantly higher rate of hits based on medical record 
review (14.60%) than the rate calculated across all MC+ MCOs (11.80%) using the Hybrid Method. 
Missouri Care reported a significantly lower rate of hits (8.33%) than the all MC+ MCO rate. 
Differences may be due to differences in the processes for carrying out medical record reviews and 


compiling hybrid data to calculate the rate. 


Figure 9 - MC+ Managed Care Program HEDIS 2005 Well-Child Visits in the First 15 Months of Life, 
Hybrid Rate Only 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 
Source: MC+ MCO HEDIS 2005 Data Submission Tool (DST). 
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Tables 26 and 27 summarize the findings of the EQRO medical record review validation and 
Attachment XII (Impact of Medical Record Findings) of the Protocol. Three of the MC+ MCOs 
(Mercy Health Plan, Missouri Care, and FirstGuard) used the Hybrid Method of calculation. Two of 
the three selected a sample of 411 eligible members, consistent with HEDIS technical specifications. 
Missouri Care selected a sample of 288 eligible members, for undetermined reasons. A total of 87 
of the 13Imedical records (66.4%) reported as hybrid hits by MC+ MCOs were sampled for 
validation by the EQRO. All 87 medical records were received for review and 56 were able to be 
validated (64.37%), resulting in an error rate of 35.63% across all MC+ MCOs using the Hybrid 
Method of calculation. The number of False Positive Records (the total amount that could not be 
validated) was 47 of the 131 reported hits. The error rate ranged from 14.81% to 60.00%. The 
estimated bias for individual MC+ MCOs based on the medical record validation ranged from a |.2% 
to 8.8% overestimate in the rate, with an average overestimate of 4.20% for all MC+ MCOs. Table 


26 shows the impact of the medical record review findings. 
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Table 26 - Impact of Medical Record Findings, HEDIS 2005 Well Child Visits in the First 15 Months of Life Measure 
MC+ = 


Item Audit Elements | HCUSA | MOCare — : 


: | NA ee: _ - 
If no, the rest of the spreadsheet will be completed to 
determine the impact on the final rate. See below See Below See Below 


Weight of Each Medical Record (Impact of each medical record on 

the final overall rate; determined by dividing 100% by the 

denominator) 0.002 0.0003 0.002 
medical records inappropriately counted as numerator positives) 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and 
the item relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper 
explanation in documentation. | = Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in 
documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number 
Applicable. 


Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Table 27 - Medical Record Validation for HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 
, i Number 


) Number Medical | Number Medical | Medical | 
Numerator Hits | Records Records Records Rate Validated} | Weight of 
Denominator oVam i (=Le [Cer] Sampled for Received for Validated by of Records Accuracy |Each Medical] 
MC+ MCO (Sample size) | (Sample Size) | AX =elo] ge [-} eau peters = Sek) Audit - oe | _ EQROU Received Rate Error Rate} AX =Tele] ge 


40k 


a 85.2% 85.2% 14.8% 0.003 


All MC+ MCOs 1,110} 131] 64.4% | 64.4% | 35.6% | 0.0009} 





Note: DST = Data Submission Tool; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); Accuracy Rate = Number of Medical Records Validated by 
the EQRO/Number of Records Selected for Audit by EQRO; Error Rate = 100% - Accuracy Rate; Weight of Each Medical Record = 100% / Denominator (Sample Size); Estimated 
Bias from Medical Records = Percent of bias due to the medical record review = False Positive Rate * Weight of Each Medical Record. 

Source: MC+ MCO Data Submission Tools (DST); BHC, Inc. 2005 External Quality Review Performance Measures Validation. 
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Table 28 shows the validation of numerators based on the review of numerator extract files and the 
medical record review. Items [3.8 through 13.13 relate to the Hybrid Method and were not 
applicable to Community CarePlus, HealthCare USA, Family Health Partners, or Blue-Advantage 
Plus of Kansas City. Across MC+ MCOs, 92.5% of the criteria for calculating numerators were met. 
Two of the three MC+ MCOs calculating the measures using the Hybrid Method (Missouri Care and 
FirstGuard) Met all criteria (100.0%) for using the appropriate data to identify the at-risk population, 
using complete medical event codes, correctly classifying members for inclusion in the numerator, 
using consistent non-standard code maps, adequately training record review staff, and using 
appropriate notation for medical record reviews for the measured event. Six of the seven MC+ 
MCOs (85.7%) Met the criteria for correctly evaluating medical event codes when classifying 
members for inclusion or exclusion in the numerator for the measure. HealthCare USA did not 
provide the EQRO with the service codes for their numerator events file, so the EQRO could not 
determine the validity of the service codes. One of the three MC+ MCOs (33.3%) using the Hybrid 
Method carried out medical record abstractions in a reliable, accurate manner. The exclusion of 
members from the medical record review by only requesting records that they expected to receive 
results in a rating of not met for Mercy Health Plan. Missouri Care Partially Met this criteria. The 
MC+ MCOs Met 60.0% to 100.0% of criteria for calculating the numerator for the HEDIS 2005 


Well-Child Visits in the First 15 Months of Life measure. 
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Table 28 - Numerator Validation Findings, HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements of @ 24 HCUSA BA+ Partially Met Met PVeye) [Cet-}e) (=) Rate Met 


The MCO/PIHP has used the appropriate data, including 
linked data from separate data sets, to identify the entire at- 
13.1 risk population. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The MCO/PIHP has in place and utilizes procedures to 
capture data for those performance indicators that could be 
easily under-reported due to the availability of services 
13.2 outside the MCO/PIHP. NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO's/PIHP's use of codes used to identify medical 
events are complete, accurate, and specific in correctly 





13.3 describing what has transpired and when. 2 2 0 2 2 2 5 6 0 1 7 85.7% 
when classifying members for inclusion or exclusion in the 

13.4 numerator. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
The MCO/PIHP has avoided or eliminated all double- counted 

13.5 members or numerator events. 2 3 3 - - 2 2 7 0 0 7 100.0% 


Any non-standard codes used in determining the numerator 
have been mapped to a standard coding scheme in a 
manner that is consistent, complete, and reproducible as 
evidenced by a review of the programming logic or a 
ele demonstration of the program. NA NA NA NA NA NA NA 0 0 0 0 NA 
Any time parameters required by the specifications of the 
performance measure are adhered to (i.e., that the 
measured event occurred during the time period specified or 
13.7 defined in the performance measure). 5 > 5 ‘ . 5 2 7 0 0 7 100.0% 


Medical record reviews and abstractions have been carried 
out in a manner that facilitates the collection of complete, 


13.8 accurate, and valid data. NA 0 NA iL NA 2 NA 1 1 il 3 33.3% 
Record review staff have been properly trained and 

13.9 supervised for the task. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Record abstraction tools require the appropriate notation 

13.10 that the measured event occurred. NA 2 NA 2 NA D NA 3 0 0 3 100.0% 
Record abstraction tools require notation of the results or 

13.11 findings of the measured event (if applicable). NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 


Data included in the record extract files are consistent with 
data found in the medical records as evidenced by a review 
of a sample of medical record for applicable performance 
measures. (From Medical Record Review Validation Tools- 


13.12 Table 5, ATTACHMENT XII) NA 2 NA ®) NA 2 NA 3 0 0 3 100.0% 


The process of integrating administrative data and medical 
record data for the purpose of determining the numerator is 


13.13 consistent and valid. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Number Met 5 10 3 10 5 11 5 49 1 3 53 92.5% 
Number Partially Met 0 0 0 1 0 0 0 
Number Not Met 0 1 2 0 0 0 0 
Number Applicable 5 11 5 11 5 11 5 
Rate Met 100.0% 90.9% 60.0% 90.9% 100.0% 100.0% 100.0% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item 
relates to the Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in 
documentation. | = Partially Met; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or 
insufficient explanation in documentation. * Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Sampling Procedures for Hybrid Method 

The objectives of this activity were to evaluate the MC+ MCOs’ ability to randomly sample from the 
eligible members for the measure when using the Hybrid Method of calculation. Table 29 
summarizes the findings of Attachment XV (Sampling Validation Findings) of the Protocol. Items 
[5.3 (Each provider had an equal chance of being sampled) and 15.9 (Documenting if the requested 
sample size exceeded the eligible population size) did not apply to any of the MC+ MCOs for this 
measure; and none of the items were applicable to Community CarePlus, HealthCare USA, Family 
Health Partners, or Blue-Advantage Plus of Kansas City. Across all MC+ MCOs, the criteria for 
sampling were Met 96.7% of the time. MC+ MCOs Met criteria (100.0%) for random sampling 
without systematic exclusion, examining files for bias, assuring there was no correlation between 
samples drawn, assuring members had the same chance of being included at baseline and follow-up 
measurement, maintaining sample files, meeting sample size requirements of the performance 
measure specifications, oversampling to accommodate for exclusions, and making substitutions 
properly. The criteria for sample exclusions was Met by 66.7% of the MC+ MCOs. Mercy Health 
Plan systematically excluded medical records sampled for the Hybrid Method by not requesting all 
the records that did not have administrative hits for medical record review. The MC+ MCOs using 
the Hybrid Method of calculating the HEDIS 2005 Well-Child Visits in the First 15 Months of Life 


measure Met 90.0% to 100.0% of the criteria for proper sampling. 


SUBMISSION OF MEASURES TO THE STATE 
Reports from the SPHA were obtained regarding the submission of the HEDIS 2005 Well-Child 
Visits measure. All MC+ MCOs reported the measure to the SPHA and SMA. 
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Table 29 - Sampling Validation Findings, HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements BA+ Partially Met Met PAVey oli fet-}e) (=) Rate Met 


Each relevant member or provider had an equal chance of 
being selected; no one was systematically excluded from 


15.1 the sampling. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Ine MCU / PIHP Ttollowed the specitications set Tortn in the 
performance measure regarding the treatment of sample 
exclusions and replacements, and if any activity took place 
involving replacements of or exclusions from the sample, 
the MCO/PIHP kept adequate documentation of that 

152 activity. NA 0 NA 2 NA 2 NA 2 0 | 3 66.7% 
Each provider serving a given number of enrollees had the 
same probability of being selected as any other provider 

15.3 serving the same number of enrollees.* NA NA NA NA NA NA NA 0 0 0 0 NA 
any bias was detected, the MCO/PIHP is able to provide 
documentation that describes any efforts taken to correct 

15.4 it. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Ihe sampling methodology employed treated all measures 
independently, and there is no correlation between drawn 

15.5 samples. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 


Relevant members or providers who were not included in 
the sample for the baseline measurement had the same 
chance of being selected for the follow-up measurement as 
qi5s6 providers who were included in the baseline. NA 2 NA ) NA 2 NA 3 0 0 3 100.0% 


The MCO/PIHP has policies and procedures to maintain files 
from which the samples are drawn in order to keep the 
population intact in the event that a sample must be re- 
drawn, or replacements made, and documentation that the 





15.7 original population is intact. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Sample sizes meet the requirements of the performance 
15.8 measure specifications. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 


The MCO/PIHP has appropriately handled the 
documentation and reporting of the measure if the 


15.9 requested sample size exceeds the population size. NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP properly oversampled in order to 
15. 10 accommodate potential exclusions NA a NA 2 NA 2) NA 3 0 0 3 100.0% 


Substitution applied only to those members who met the 
exclusion criteria specified in the performance measure 

15.11 definitions or requirements. NA 2 NA 2 NA 2 NA 3 0 0 3 100.0% 
Substitutions were made for properly excluded records and 


15.12 the percentage of substituted records was documented. NA NA 2 NA 2 NA 8 0 0 3 100.0% 


Number Met 












Number Partially Met 
Number Not Met 
Number Applicable 
Rate Met 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to the 
Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially Met; 0 
= Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in documentation. * Item is 
not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 





Pertormasce Mass goment 


Performance Management Solutions Group 99 
roe teaestencrnan mB A division of Behavioral Health Concepts, Inc. 


Missouri MC+ Managed Care External Quality Review Section 3 
Report of Findings — 2005 Validation of Performance Measures 


Final Validation Findings 

Table 30 shows the final data validation findings for the calculation of the HEDIS 2005 Well-Child 
Visits in the First 15 Months of Life measure and the total estimated bias in calculation based on the 
validation of medical record data and review of the MC+ MCO extract files. Figure |0 illustrates 
the differences between the rates reported to the SPHA and those calculated by the EQRO. The 
rate for all MC+ MCOs calculated based on data validated by the EQRO was 36.36%, while the rate 
reported by MC+ MCOs was 38.42%, a 2.06% overestimate. 


Table 30 - Final Data Validation for HEDIS 2005 Well-Child Visits in the First 15 Months of Life Measure 
Percentage of 
Medical 
Administrative Record Hits Total Hits Total 
Hits Validated by) Validated by Validated by Rate Reported | Validated by! Estimated 
| (Ot am Wy (Le) |e] sXe) EQRO* |e) fe) by MCO (DST) 
Community Care Plus NA 32.60% 32.57% 
Blue Advantage Plus NA 33.06% 30.06% 


Family Health Partners NA 23.91% 23.85% 
FirstGuard 40.00% 51.58% 57.18% 


HealthCare USA (all 3 regions) NA 41.50% 39.05% 
Mercy Health Plan 70.00% 43.80% 40.36% 
Missouri Care 85.20% 75.69% 75.50% 
All MC+ MCOs 





Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral Health 
Concepts, Inc.); Administrative/Medical Record Hits Validated by EQRO = Hits the EQRO was able to reproduce from the 
data provided by the MCO; Total Hits Validated by EQRO = Administrative Hits Validated by EQRO + Medical Record Hits 
Validated by EQRO; False Positive Records = Error Rate * Rate Reported by MCO; Rate Validated by EQRO = Total Hits 
Validated by EQRO / Denominator (DST); Total Estimated Bias = Rate Reported by MC+ MCO -— Rate Validated by EQRO. 
Positive numbers represent an overestimated by the MCO. 


*For a more detailed explanation of how Medical Record Hit percentages were calculated, please see Table 27. 


Sources: MC+ Managed Care Organization HEDIS 2004 data Submission Tools (DST); BHC, Inc. External Quality 
Review Performance Measure Validation. 
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Figure 10 - Rates Reported by MC+ MCOs and Validated by EQRO, HEIDS 2005 Well-Child Visits in the 
First 15 Months of Life Measure 
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CCP BA+ FHP HCUSA MHP MOCare All MC+ 
MCOs 
MC+ MCO 
@ Rate Reported by MCO (DST) 35 gu, 33.06% 23.91% 51.58% 41.50% 43.80% 75.69% 38.42% 
O Rate Validated by EQRO 32.57% 30.06% 23.85% 57.18% 39.05%** 40.36% 75.50% 36.82% 


Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); BHC, Inc., 2005 External Quality Review 
Performance Measure Validation. *Rate calculated by EQRO is based on data provided to the EQRO for 
review; data provided could not be independently validated. 
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HEDIS 2005 ANNUAL DENTAL VISIT 

Data Integration and Control 

The objective of this activity was to assess the MC+ MCOs’ ability to link data from multiple 
sources. It is based on the integrity of the management information systems and the ability to 
ensure accuracy of the measures. For the HEDIS 2005 Annual Dental Visit measure, the sources of 
data included enrollment, eligibility, and claim files. Table 31 summarizes the findings of Attachment 
V (Data Integration and Control Findings) of the Protocol. The rate of items that were Met was 


calculated across MC+ MCOs and from the number of applicable items for each MC+ MCO. 


Across all MC+ MCOs, 98.9% of the criteria were Met for having accurate and established 
procedures for transferring data into data repositories for calculation of the measure, coordinating 
data integration with vendors, and following standards associated with programming and testing. 
One MC+ MCO (Mercy Health Plan) Partially Met the criteria for retaining copies of files or 
databases used for performance measure reporting, due to a system change the MCO could not 
produce the database used for HEDIS 2005 calculations, they assured the EQRO that it was available 
on back-up tape. Each MC+ MCO calculating the measure Met 92.3% to 100.0% of the criteria for 


data integration and control. 
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Table 31 - Data Integration and Control Findings, HEDIS 2005 Annual Dental Visit 


| Oo | O10) All MC+ MCOs 





Number Number Not Number 
Audit Elements BA+ Partially Met Met 7Vo) oe) [er-} o) (=) Rate Met 


MCO/PIHP processes accurately and completely transfer data 
from the transaction files (e.g., membership, provider, 
encounter/claims) into the repository used to keep the data 
until the calculations of the performance measures have been 


5.1 completed and validated. 2 2 2 2 2 2 2 a 0 0 7 100.0% 


52 Samples of data from repository are complete and accurate. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


MCO’'s/PIHP’s processes to consolidate diversified files, and to 
extract required information from the performance measure 
5.3 repository are appropriate. 2 2 2 2 2 2 2 7 0 6) 7 100.0% 
Actual results of file consolidations or extracts were consistent 
with those which should have resulted according to 
54 documented algorithms or specifications. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


Procedures for coordinating the activities of multiple 
subcontractors ensure the accurate, timely, and complete 
5.5 integration of data into the performance measure database. 2 2 , ? ? ? 2 7 0 0 7 100.0% 


Computer program reports or documentation reflect vendor 
coordination activities, and no data necessary to performance 
measure reporting are lost or inappropriately modified during 


5.6 transfer. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The repository’s design, program flow charts, and source 
5.7 codes enable analyses and reports. 2 2 2 5 2 2 2 7 0 0 7 100.0% 


Proper linkage mechanisms have been employed to join data 
from all necessary sources (e.g., identifying a member with a 
5.8 given disease/ condition). 2 2 2 2 2 2 2 7 0 0 7 100.0% 
Examine and assess the adequacy of the documentation 
governing the production process, including MCO/PIHP 
production activity logs, and MCO/PIHP staff review of report 
5.9 runs. 2 2 2 2 2 2 2 7 0 0 7 100.0% 


5.10 Prescribed data cutoff dates were followed. 2 5 2 2 O 2 2 7) 0 0 7 100.0% 


The MCO/PIHP has retained copies of files or databases used 
for performance measure reporting, in the event that results 
5.11 need to be reproduced. 2 1 2 2 2 2 2 6 1 0 7 85.7% 
Review documentation standards to determine the extent to 
which the reporting software program is properly documented 
with respect to every aspect of the performance measurement 
reporting repository, including building, maintaining, 
5.12 managing, testing, and report production. 2 2 2 2 2 2 5 7) 0 0 7 100.0% 


Review the MCO’s/PIHP’s processes and documentation to 
determine the extent to which they comply with the MCO/PIHP 
standards associated with reporting program specifications, 


5.13 code review, and testing. a 2 2 2 2 2 2 7 0 0 7 100.0% 

Number Met 13 12 13 13 13 13 13 90 1 0 91 98.9% 
Number Partially Met 0 1 0 0 0 0 0 
Number Not Met 0 0 0 0 0 0 0 
Number Applicable 13 13 13 13 13 13 13 
Rate Met 100.0% 92.3% 100.0% 100.0% 100.0% 100.0% 100.0% 

Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to the 

Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially Met; 


0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in documentation. * 
Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Documentation of Data and Processes 

The objectives of this activity were to assess the documentation of data collection; the process of 
integrating data into a performance measure set; the procedures used to query the data set for 
sampling, numerators and denominators; and the ability to apply proper algorithms. Table 32 
summarizes the findings of Attachment VI (Data and Processes Used to Calculate and Report 
Performance Measures) of the Protocol. Items 7.3 (statistical testing of results and corrections 
made after processing), 7.7 (sampling techniques), 7.9 (data consistency from measure to measure), 
and 7.10 (appropriate statistical functions for confidence intervals) did not apply to the measure. 
Across all MC+ MCOs, 74.3% of criteria for calculating and reporting performance measures were 
Met. All MC+ MCOs Met the criteria for following data file and field definitions and the integration 
of external data (100.0%). The criteria: “demonstration of detailed queries for identifying eligible 
members” was met by six of the seven MC+ MCOs (85.7%). HealthCare USA did not meet this 
criteria as they did not provide the service code information necessary for the EQRO to determine 
that the population was in fact eligible for the measure. Although MC+ MCOs frequently graphed 
the rates of performance over several years, only 42.9% of the MC+ MCOs used the calculation of 
Statistical significance in rates from year to year as a measure of the significance of fluctuation in the 
measure (see items 7.8 and 7.11). Each MC+ MCO Met 60.0% to 100.0% of the criteria for 


calculating and reporting performance measures. 
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Table 32 - Data and Processes Used to Calculate and Report Performance Measures, HEDIS 2005 Annual Dental Visit Measure 


MC+ MCO | All MC+ MCOs 






Number Number Not Number 
Item Audit Elements MHP HCUSA IW Cot er-T a=) A+ Number Met} Partially Met) Met PVs) e) [(et-] 0) (= Rate Met 


Statistical testing of results and any corrections or 
7.3 adjustments made after processing. NA NA NA NA NA NA NA NA 


Detailed medical record review methods and practices, 
NA NA NA NA NA NA NA NA 


including the qualifications of medical record review supervisor 


and staff; reviewer training materials; audit tools used, 
including completed copies of each record-level reviewer 
when aetermining improvement In performance between 
measurement periods, appropriate statistical methodology is 
7.11 applied to determine levels of significance of changes. 42.9% 


determination; all case-level critical performance measure 
data elements used to determine a positive or negative event 
or exclude a case from same; and inter-rater reliability 
testing procedures and results. 

Number Partially Met 0 0 0 0 2 
Number Not Met 2 F 1 2 0 

Rate Met 60.0% 60.0% 80.0% 100.0% 60.0% 100.0% 60.0% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to the 
Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially Met; 
0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in documentation. * 
Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 







































If sampling used, description of sampling techniques, and 
documentation that assures the reviewer that samples used 
for baseline and repeat measurements of the performance 

measures were chosen using the same sampling frame and 
methodology. 























Data that are related from measure to measure are consistent 
(e.g., membership counts, provider totals, number of 
pregnancies and births). 
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Processes Used to Produce Denominators 

The objective of this activity was to determine the extent to which all eligible members were 
included in the denominator, evaluate the programming and logic source codes, and evaluate the 
specifications for each measure. For the HEDIS 2004 Annual Dental Visits measure, the sources of 
data include enrollment, eligibility, and claim files. Table 33 summarizes the findings of Attachment 
X (Denominator Validation Findings) of the Protocol. Items 10.5 (Identification of gender of the 
member), 10.6 (Calculation of member months or years), and 10.10 (Systems for estimating 
populations when they are unable to accurately be counted) were not applicable to this measure. 
Across all MC+ MCOs, 98.0% of criteria for calculating and reporting performance measures were 
Met. Six out of seven MC+ MCOs Met all the criteria for producing denominators according to 
specifications, one MC+ MCO (HealthCare USA) did not meet the requirement of “properly 
evaluating the completeness and accuracy of any codes used to identify medical events, such as 
diagnoses, procedures, or prescriptions, and these codes have been appropriately identified and 
applied as specified in each performance measure’ due to the MC+ MCO not providing the codes 
to the EQRO for review. Each MC+ MCO Met 85.7% to 100.0% of the criteria for the process used 


to produce denominators. 
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Table 33 - Denominator Validation Findings, HEDIS 2005 Annual Dentist Visit Measure 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements CCP HCUSA MOCare BA+ Partially Met Met PVoye) [ret= le) (=) Rate Met 


All members who were eligible to receive the specified services 
were included in the initial population from which the final 
denominator was produced. This "at risk" population included 
both members who received the services, as well as those who 
did not. This same standard applies to provider groups or other 
relevant populations identified in the specifications of each 


10.1 performance measure. 2 2 2 2 2 2 Z 7 0 0 7 100.0% 


For each measure, programming logic or source code which 
identifies, tracks, and links member enrollment within and 
across product lines (e.g., Medicare and Medicaid), by age and 
sex, aS well as through possible periods of enrollment and 
disenrollment, has been appropriately applied according to the 
10.2 specifications of each performance measure. 2 2 2 a 5 2 > 7 0 0 7 100.0% 





Calculations of continuous enrollment criteria were correctly 
10.3 carried out and applied to each measure (if applicable). 2 2 2 9 2 a 5 7 0 0 7 100.0% 


Proper mathematical operations were used to determine patient 
10.4 age or range. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The MCO/PIHP can identify the variable(s) that define the 
member's sex in every file or algorithm needed to calculate the 
performance measure denominator, and the MCO/PIHP can 
explain what classification is carried out if neither of the required 


10.5 codes is present.* NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO/PIHP has correctly calculated member months and 
10°6 member years, if applicable to the performance measure.* NA NA NA NA NA NA NA 0 0 0 0 NA 


The MCO/PIHP has properly evaluated the completeness and 
accuracy of any codes used to identify medical events, such as 
diagnoses, procedures, or prescriptions, and these codes have 
been appropriately identified and applied as specified in each 


10.7 performance measure. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
Any time parameters required by the specifications of the 
performance measure are followed (e.g., cut off dates for data 
collection, counting 30 calendar days after discharge from a 


10.8 hospital, etc.). 2 2 2 2 D 2 2 7 0 0 z; 100.0% 


Performance measure specifications or definitions that exclude 
members from a denominator were followed. For example, if a 
measure relates to receipt of a specific service, the denominator 
may need to be adjusted to reflect instances in which the patient 
10.9 refuses the service or the service is contraindicated. 2 2 a a 2 2 a 7 0 0 7 100.0% 


Systems or methods used by the MCO/PIHP to estimate 
populations when they cannot be accurately or completely 





10.10 counted (e.g., newborns) are valid.* NA NA NA NA NA NA NA 0 0 0 0 NA 

Number Met 7 z 6 7 7 7 7 48 (0) 1 49 98.0% 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 0 1 0 0 0 0 
Number Applicable 7 7 7 7 7 7 7 
Rate Met 100.0% 100.0% 85.7% 100.0% 100.0% 100.0% 100.0% 

Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to the 

Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially Met; 


0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in documentation. * 
Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Figure II illustrates the rate of eligible members per MC+ MCO based on the enrollment of all 
MC+ MCO Waiver Recipients as of December 31, 2004, the end of the CY2004 measurement year. 
It was expected that MC+ MCOs would identify similar proportions of eligible members for the 
measure. The rate of eligible members (percent of eligible members divided by the total 
enrollment) was calculated for all MC+MCOs. Two-tailed z-tests of each MC+ MCO comparing 
each MC+ MCO to the state rate of eligible members for all MC+ MCOs were calculated at the 
95% level of confidence. HealthCare USA and Community CarePlus identified significantly higher 
rates of eligible members (51.48% and 49.13% respectively) compared to the rate for all MC+ 
MCOs (47.54%), while Mercy Health Plan identified a significantly lower rate of eligible members 


(39.78%). This may be a function of the claims administration process or member characteristics. 


Figure || - MC+ Managed Care Program HEDIS 2005 Annual Dental Visit, Eligible Members 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. Enrollment as of the last week in December 2004 (the measurement year) was used to 
calculate the rate. 

Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); Missouri Department of Social Services, Division 
of Medical Services, State MPRI Session Screens, enrollment figures for all Waivers, December 31, 2004. 
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Processes Used to Produce Numerators 

The objectives of this activity were to evaluate the MC+ MCOs ability to accurately identify medical 
events, evaluate the ability to identify events from other sources, evaluate procedures for non- 
duplicate counting of multiple events, review time parameters and the use of non-standard code 
maps, and assess the processes and procedures for collecting and incorporating medical record 
review data. For the HEDIS 2005 Annual Dental Visits measure, the procedures for the Hybrid 
Method did not apply, as HEDIS 2005 technical specifications allow only for the use of the 


Administrative Method of calculating the measure. 


Table 34 shows the numerators, denominators, rates, and confidence intervals submitted by the 
MC+ MCOs to the SPHA on the DST. The rate for all MC+ MCOs was calculated by the EQRO, 
thus there is no confidence interval reported for the statewide rate. The reported rate for all MC+ 
MCOs was 29.76% and the rate validated by the EQRO was 29.82%, a 0.06% underestimate. Figure 
|2 illustrates the rates reported by the MC+ MCOs. The rate reported by each MC+ MCO was 
compared with the rate for all MC+ MCOs, with two-tailed z-tests conducted at the 95% 
confidence interval to compare each MC+ MCO with the rate for all MC+ MCOs. The rate for all 
MC+ MCOs was lower than the National Medicaid rate (42.70%). All MC+ MCOs reported a rate 
lower than the National Medicaid rate. Only Family Health Partners reported a rate (39.10%) that 
was comparable to the National Medicaid rate. Family Health Partners and Blue-Advantage Plus of 


Kansas City reported rates significantly higher than the rate for all MC+ MCOs (39.10% and 33.80% 


respectively), while Mercy Health Plan reported a rate (20.52%) significantly below the rate for all 
MC+ MCOs. 
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Table 34 - Data Submission and Final Data Validation for HEDIS 2005 Annual Dental Visit Measure (Combined Ages) 


Eligible Member Administrative Administrative Rate Total 
Population Reported by) Hits Reported by/ Hits Validated by} Rate Reported | Validated by! Estimated 
MC+ MCO MCO (DST) (ole @ Pha B) [0] 30) | by MCO (DST) | EQRO Bias 


olen Care Plus 22621 7043 7043 a1, 13% 31.13% 0.00% 
Family + Health: Parneis 24342 9517 9516 39.10% 39.09% 0.00% 


HealthCare USA aul 3 regions) 95309 27536 27679 28.89% 29.04% -0.15% 
14900 4271 4270 28.66% 28.66% 0.01% 


All MC+ MCOs 210083 62517 62650 29.76% 29.82% -0.06% 





Note: DST = Data Submission Tool; NA = Not Applicable; EQRO = External Quality Review Organization (Behavioral Health Concepts, Inc.); LCL = 95% Lower 
Confidence Limit; UCL = 95% Upper Confidence Limit. Rate Validated by EQRO = Administrative Hits Validated by EQEO / Eligible Population. Estimated Bias = Rate 
Reported by MCO (DST) - Rate Validated by EQRO. Positive bias indicates an overestimate. 

Source: MC+ Managed Care Organization HEDIS 2005 Data Submission Tools (DST). 
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Figure 12 - MC+ Managed Care Program HEDIS 2005 Annual Dental Visit, Rates 
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Note: Error bars on the y-axis represent 95% confidence intervals; * Indicates values are significant at the 95% level of 
significance, two-tailed z-test. 

Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); National Committee for Quality Assurance 
(NCQA). 


Table 35 shows the validation of numerators based on the review of numerator extract files and the 
medical record review. Item |3.2 was not applicable to the HEDIS 2005 Annual Dental Visit 
measure. Items [3.8 through 13.13 relate to the Hybrid Method of calculation and were not 
applicable to the measure. Across all MC+ MCOs, 94.3% of the criteria for calculating numerators 
were met. All MC+ MCOs Met criteria for using the appropriate data to identify the at-risk 
population, avoiding double-counting of events, and following time parameters specified for the 
measure. Six of the seven MC+ MCOs (85.7%) Met the criteria for using complete medical event 
codes and correctly classifying members for inclusion in the numerator. HealthCare USA did not 
provide service codes to the EQRO so that the calculation of the numerator could be independently 


validated. The MC+ MCOs Met 60.0% to 100.0% of criteria for the calculation of the numerator. 
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Table 35 - Numerator Validation Findings, HEDIS 2005 Annual Dental Visit Measure 


MC+ MCO All MC+ MCOs 


Number Number Not Number 
Audit Elements CCP HCUSA BA+ Partially Met Met PVeye) [Cet-}e) (=) Rate Met 


The MCO/PIHP has used the appropriate data, including 
linked data from separate data sets, to identify the entire at- 
13.1 risk population. 2 2 2 2 2 2 2 7 0 0 7 100.0% 
The MCO/PIHP has in place and utilizes procedures to 
capture data for those performance indicators that could be 
easily under-reported due to the availability of services 
13.2 outside the MCO/PIHP. NA NA NA NA NA NA NA 0 0 0 0 NA 
The MCO's/PIHP's use of codes used to identify medical 
events are complete, accurate, and specific in correctly 





13.3 describing what has transpired and when. 2 2 0 5 3 2 2 6 0 1 7 85.7% 
when classifying members for inclusion or exclusion in the 

13.4 numerator. 2 2 0 2 2 2 2 6 0 1 7 85.7% 
The MCO/PIHP has avoided or eliminated all double- counted 

13.5 members or numerator events. 2 2 2 2 2 2 2 7 0 re) 7 100.0% 


Any non-standard codes used in determining the numerator 
have been mapped to a standard coding scheme in a 
manner that is consistent, complete, and reproducible as 
evidenced by a review of the programming logic or a 
ate demonstration of the program. NA NA NA NA NA NA NA 0 0 0 0 NA 
Any time parameters required by the specifications of the 
performance measure are adhered to (i.e., that the 
measured event occurred during the time period specified or 
13.7 defined in the performance measure). g, , 2 2 2 2 5 7 0 0 7 100.0% 


Medical record reviews and abstractions have been carried 
out in a manner that facilitates the collection of complete, 


13.8 accurate, and valid data. NA NA NA NA NA NA NA 0 0 0 0 NA 
Record review staff have been properly trained and 

13.9 supervised for the task. NA NA NA NA NA NA NA 0 0 0 0 NA 
Record abstraction tools require the appropriate notation 

13.10 that the measured event occurred. NA NA NA NA NA NA NA 0 0 0 0 NA 
Record abstraction tools require notation of the results or 

13.11 findings of the measured event (if applicable). NA NA NA NA NA NA NA 0 0 0 0 NA 


Data included in the record extract files are consistent with 
data found in the medical records as evidenced by a review 
of a sample of medical record for applicable performance 
measures. (From Medical Record Review Validation Tools- 


ley) Table 5, ATTACHMENT XII) NA NA NA NA NA NA NA 0 0 0 0 NA 


The process of integrating administrative data and medical 
record data for the purpose of determining the numerator is 











13.13 consistent and valid. NA NA NA NA NA NA NA 0 0 0 0 NA 
Number Met 5 5 3 5 5 5 5 33 ft) 2 E15) 
Number Partially Met 0 0 0 0 0 0 0 
Number Not Met 0 0 2 0 0 0 0 
Number Applicable 5 5 5 5 5 5 5 
Rate Met 100.0% 100.0% 60.0% 100.0% 100.0% 100.0% 100.0% 
Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative Method was used by the MC+ MCO and the item relates to the 
Hybrid Method; 2 = Met and validated through HEDIS software certification process and proper explanation in documentation or proper explanation in documentation. | = Partially Met; 


0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the process in documentation or no or insufficient explanation in documentation. * 
Item is not applicable to the measure being validated. Rate Met = Number Met / Number Applicable. 
Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation 
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Figure 13 - Rates Reported by MC+ MCOs and Validated by EQRO, HEIDS 2005 Annual Dental Visit 
Measure 
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HCUSA MOCare All MC+ 
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MC+ MCO 


Rate Reported by MCO (DST) —_ 34.43% 33.80% 39.10% 27.07% 28.89% 20.52% 28.66% 29.76% 


O Rate Validated by EQRO 31.13% 33.79% 39.09% 27.07% 29.04%" 20.49% 28.66% 29.82% 





Sources: MC+ MCO HEDIS 2005 Data Submission Tool (DST); BHC, Inc., 2005 External Quality Review Performance 
Measure Validation. * Rate calculated by EQRO is based on data provided to the EQRO for review; data provided could 
not be independently validated. 


Submission of Measures to the State 

Reports from the SPHA were obtained regarding the submission of the HEDIS 2005 Annual Dental 
Visit Measure. All seven MC+ MCOs calculated and submitted the measure to the SPHA and SMA. 
The rate reported by MC+ MCOs ranged from 20.52% to 39.10%. The rate of all MC+ MCOs 
calculated based on data validated by the EQRO was 29.82%, consistent with the rate reported by 
MC+ MCOs (29.76%), with no observed bias. 


Final Validation Findings 

Tables 36 through 38 provide summaries of ratings across all Protocol Attachments for each MC+ 
MCO and measure validated. The rate of compliance with the calculation of the Childhood 
Immunization Status, Combination #2 measure specifications ranged from 82.35% (HealthCare USA) 
to 100% (FirstGuard and Missouri Care), with a rate of 93.46% across all MC+ MCOs and items. 
For the calculation of the Well-Child Visits in the First 15 Months of Life measure, MC+ MCO 
compliance with specifications ranged from 100.0% (First Guard) to 86.21% (HealthCare USA), with 
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a rate of 93.85% across all MC+ MCOs and items. For the rate of compliance with specifications for 
the calculation of the Annual Dental Visits measure, the rate ranged from 100.0% (First Guard and 


Missouri Care) to 86.67% (HealthCare USA), with an average of 93.33% across all MC+ MCOs. 


Table 36 - Summary of Attachment Ratings, HEDIS 2005 Childhood Immunization Status, Combination #2 
Measure 





All Audit All MC+ MCOs 





SSR Ine | [CS Se) | LS EE) tele] [alae [Pe AS) eae] 
Number Met 48 47 28 49 49 49 30 
Number 
Partially Met 0 1 ) 0 0 0 2 3 
Number NOT 
Met 3 3 6 2 2 2 0 18 
Number 
Applicable 51 51 34 51 51 51 32 321 
Rate Met 94.12% 92.16% 82.35% 96.08% 96.08% 96.08% 93.75% 93.46% 


Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative 
Method was used by the MC+ MCO and the item relates to the Hybrid Method; 2 = Met and validated through HEDIS 
software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the 
process in documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure 
being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 


Table 37 - Summary of Attachment Ratings, HEDIS 2005 Well-Child Visits in the First 15 Months of Life 
Measure 





All Audit All MC+ MCOs 





Elements | CCP | MHP | HCUSA | MOCare | FHP | FG | BAt+ | 
Number Met 27 43 25 47 27 48 27 
Number 









Partially Met 0 1 0 1 0 0 D 4 
Number NOT 


Applicable 29 48 29 48 29 48 29 260 
89.58% 86.21% 97.92% 93.10% 100.00% 93.10% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative 
Method was used by the MC+ MCO and the item relates to the Hybrid Method; 2 = Met and validated through HEDIS 
software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the 
process in documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure 
being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 
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Table 38 - Summary of Attachment Ratings, HEDIS 2005 Annual Dental Visit Measure 
All Audit All MC+ MCOs 





Elements 
Number Met 
NUumMper 


Partially Met 0 1 0 0 0 0 2 3 
NUMber NOT 









Applicable 30 30 30 30 30 30 30 210 
90.00% 83.33% 100.00% 93.33% 100.00% 





Note: A = Administrative; H = Hybrid; NA = Not Applicable to the method employed by the MC+ MCO; Administrative 
Method was used by the MC+ MCO and the item relates to the Hybrid Method; 2 = Met and validated through HEDIS 
software certification process and proper explanation in documentation or proper explanation in documentation. | = 
Partially Met ; 0 = Not Met, validated through HEDIS software certification process, but no proper explanation of the 
process in documentation or no or insufficient explanation in documentation. * Item is not applicable to the measure 
being validated. Rate Met = Number Met / Number Applicable. 

Source: BHC, Inc. 2005 External Quality Review Performance Measure Validation. 


Table 39 summarizes the final audit ratings for each of the performance measures and MC+ MCOs. 
The final audit findings for each of the measures was based on the evaluation of processes for 
calculating and reporting the measures, medical record review validation findings, and MC+ MCO 
extract files from repositories. The ratings were based on the impact of medical record review 
findings and the degree of overestimation of the rate as validated by the EQRO. The calculation of 
measures was considered invalid if the specifications were not properly followed, or if the rate 
validated by the EQRO fell outside the confidence intervals for the measure reported by the MC+ 
MCOs on the DST. 
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Table 39 - Summary of EQRO Final Audit Ratings, HEDIS 2005 Performance Measures 
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MC+ MCO Status, Combination #2 | First 15 Months of Life Annual Dental Visit 
Community Care Plus Not Valid Fully Compliant Fully Compliant 
Mercy Health Plan Substantially Compliant Substantially Compliant Fully Compliant 
HealthCare USA Not Valid Not Valid Not Valid 
Missouri Care Substantially Compliant Substantially Compliant Fully Compliant 
Family Health Partners Substantially Compliant Substantially Compliant Fully Compliant 
FirstGuard Substantially Compliant Substantially Compliant Fully Compliant 
Blue Advantage Plus Not Valid Substantially Compliant Fully Compliant 


Note: Not Valid = Measure deviated from Sate (SMA and SPHA) specifications such that the reported rate was significantly 
biased. This designation is also assigned to measures for which data provided to the EQRO could not be independently 
validated.. Significantly biased was defined by the EQRO as being outside the 95% confidence interval of the rate reported 
by the MC+ MCO on the HEDIS 2004 Data Submission Tool; Substantially Compliant = Measure was substantially 
complaint with State (SMA and SPHA) specifications and had only minor deviations that did not significantly bias the 
reported rate; Fully Compliant = Measure was fully compliant with State (SMA and SPHA) specifications. Data from Health 
Care USA was aggregated across all three regions of operation to provide MCO to MCO comparisons. 

Source: BHC Inc., 2005 External Quality Review Performance Measure Validation. 


For the HEDIS 2005 Childhood Immunization Status, Combination #2 measure, four MC+ MCOs 
(Family Health Partners, FirstGuard, Mercy and Missouri Care) were Substantially Compliant with 
the measure specifications. The rates calculated by Community CarePlus and Blue-Advantage Plus 
of Kansas City calculated by the EQRO were outside the range of the confidence intervals reported 
by these MC+ MCOs. Community CarePlus did not incorporate administrative data from the 
SPHA, and this likely accounted for 8-25% of the error associated with calculating the measure. 
HealthCare USA did not supply the EQRO with service codes (CPT Codes or ICD-9-CM Codes) as 
required by the HEDIS Technical Specifications, therefore the EQRO does not have confidence in 
the validation rates calculated for HealthCare USA. HealthCare USA supplied the EQRO with 
information that had already been processed once by their data system. The information provided 
did not include the CPT Codes or ICD-9-CM Codes, but a category or description of what services 
those service codes represent. It is necessary for the EQRO to receive service codes in the CPT or 
ICD-9-CM format not only because that is what was requested, but also because those formats are 
the industry accepted standard and the only way for the EQRO to ensure that what they are 
matching between plans is the same information. The Childhood Immunization Status, Combination 
#2 measure requires data from 2003 or later, after the full implementation of the MC+ Managed 
Care Program. However, three of the seven plans received a “Not Valid” for this measure and as 
such, this measure does not provide a valid index of performance of the MC+ Managed Care 


Program. 
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During the course of examining MC+ MCO extract files, approximately 8% to 25% of administrative 
hits for immunizations were captured from the MOHSAIC system. Although the level of 
completeness of the Registry is unknown, it does serve as a useful source for contributing to a more 


complete accounting of immunizations for the calculation of this measure. 


Six MC+ MCOs were Substantially or Fully Compliant with the specifications for calculating the 
HEDIS 2005 Well-Child Visits in the First 15 Months of Life measure. One MC+ MCO (HealthCare 
USA) did not supply the EQRO with service codes (CPT Codes or ICD-9-CM Codes) as required 
by the HEDIS Technical Specifications. HealthCare USA provided information to the EQRO 
regarding the services, but did not provide the service codes or cross reference the service codes, 
because the information provided was not in the “purest” form possible (as required by the CMS 
protocols), the EQRO does not have confidence in the validation rates calculated for HealthCare 
USA. Mercy Health Plan did not follow specifications for the Hybrid Method by systematically 
excluding medical records from review, Mercy did not request for review all medical records that 
did not produce “hits” through administrative data alone. Mercy only requested the records they 
believed they “would receive from providers”, thereby reducing the number of possible hybrid hits 
from the start of the Medical Record Review process. This is unrelated to the rate of medical 
records received for validation. There was some difference found by the EQRO in the rates 
reported by the MC+ MCOs and validated by the EQRO that was attributable to the interpretation 
of the HEDIS Technical Specifications. In the “Continuous Enrollment” requirements for 
determining the eligible population for this measure, the Technical Specifications specify the 
calculation of the child’s |5-month birthday “‘as the child’s first birthday plus 90days.” However, the 
example detailed below the instruction specifies “...a child born on January 9, 2003, and included in 
the rate of six or more well-child visits must have had six well-child visits by April 9, 2004.” This 
example equates to three calendar months, not 90 days as the “‘calculation” specifies. This 
difference in calculation is detailed in the individual MC+ MCO Performance Measures sections, 
later in this report. The EQRO chose to count “hits” for each method of calculation as it is unclear 
which method of interpretation is correct, however, the difference in calculation was not significant. 
Because the statewide rate for the Adolescent Well-Care Visits measure does not vary significantly 
from the rate validated by the EQRO, it is determined that the statewide rate does provide a valid 


index of performance of the MC+ Managed Care Program. 
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The HEDIS 2005 Annual Dental Visit measure was calculated according to specifications by nearly all 
the MC+ MCOs. Again as detailed earlier, one MC+ MCO (HealthCare USA) did not supply the 
EQRO with service codes (CPT Codes or ICD-9-CM Codes) as required by the HEDIS Technical 
Specifications, therefore the EQRO does not have confidence in the validation rates calculated for 
HealthCare USA. However, the EQRO believes that the rates calculated represent a valid index of 


performance of the MC+ Managed Care Program for MC+ MCO comparisons. 
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3.5 Conclusions 


STRENGTHS 


|. Six of seven MC+ MCOs were Fully Compliant with the specifications for calculation of the 
HEDIS 2005 Annual Dental Visit measure which represents a valid measure of performance of 
the MC+ Managed Care Program. 


2. One MC+ MCOs (Community CarePlus) was Fully Compliant with specifications for the HEDIS 
2005 Well-Child Visits in the First 15 Months of Life. 


3. Missouri Care was Substantially Compliant with the calculation of the Well-Child Visits measure 
and the rate exceeded the National Medicaid and National Commercial rates for this measure. 


4. Missouri Care was Substantially Compliant with the calculation of the Childhood Immunization 
Status, Combination #2 measure and the rate exceeded the National Medicaid rates for this 
measure. 


5. Five MC+ MCOs (Blue-Advantage Plus of Kansas City, Family Health Partners, FirstGuard, 
Mercy Health Plan, and Missouri Care) were Substantially Compliant and one MC+ MCO (CCP) 
was Fully Compliant with the specifications for calculating the HEDIS 2005 Well-Child Visits 
Measure. 


6. FirstGuard’s rate for the HEDIS 2005 Well-Child Visits measure exceeded the National 
Medicaid rate. 


7. In calculating the measures, MC+ MCOs have adequate management information systems for 
capturing and storing enrollment, eligibility, and claims information for the calculation of the 
three HEDIS 2005 measures validated. 


8. There was good integration of multiple data sources for rate calculation. 


9. Among MC+ MCOs, there was generally good documentation of the HEDIS 2005 rate 
production process. 


10. Several MC+ MCOs (Missouri Care, Mercy Health Plan, and Community CarePlus) responded 
to the preliminary findings of the performance measure validation by identifying corrective 
actions plans for the next reporting cycle. 


[1. The rates of medical record submission for the two measures utilizing the Hybrid Methodology 
were excellent, all MC+ MCOs submitted between 95% to 100% of the records requested. 


AREAS FOR IMPROVEMENT 


|. For one MC+ MCO (HealthCare USA) every measure was found to be Not Valid. This was due 
to the MC+ MCO not providing the EQRO with the data requested in its “raw” form, as 
required by the Protocols. The MCO supplied data with service codes that had already been 
translated to event names and were not in the ICD-9-CM Code or CPT Code format as 
requested or required by the Protocols. HealthCare USA has indicated that they are able to 
supply the information in the format requested and chose not to do so as it would take 
significant time, the EQRO expects to receive the data as requested and to be able to validate 
their measures in the future. 


2. The HEDIS 2004 Childhood Immunization Status, Combination #2 measure was unable to be 
validated for three of the seven MC+ MCOs and does not represent a valid measure of 
performance for the MC+ Managed Care Program. 
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One MC+ MCO (Mercy Health Plan) did not follow the Hybrid Method specifications for the 
calculation of the HEDIS 2005 Adolescent Immunization Status, Combination #2 and Well-Child 
Visits measure, resulting in ratings of Not Valid for the measures. 


Those MC+ MCOs who are not assessing the statistical significance of changes in performance 
measures over time, in order to validate stability or change from year to year, should 
incorporate statistical analysis into their HEDIS plans. 


RECOMMENDATIONS 


The SMA and SPHA should continue to support efforts to improve the utility and functionality 
of the State Public Health Immunization Registry (MOHSAIC) as well as encourage public, 
private, and non-profit providers of immunizations to use the Registry so as to obtain complete 
information about the level of care provided to MC+ Managed Care Members for the 
administration of immunizations. 


For the calculation of the HEDIS Childhood Immunization Status, Combination #2 measure, the 
Hybrid Method and the incorporation of MOHSAIC data should be required by the SMA to 
facilitate accurate and valid MC+ MCO comparisons and a valid statewide rate for comparison 
of performance with other states. 


The SMA should encourage technical assistance regarding the calculation of HEDIS performance 
measures, medical record review processes, and the capture of State Public Health 
Immunization Registry data for the calculation of performance measures. 


The SMA should re-validate measures for which all MC+ MCOs were not Fully or Substantially 
Compliant on the calculation of measures and in order to determine the impact on contract 
performance. 


It is recommended that MC+ MCOs retain medical record review data, records, and electronic 
files used to calculate and report measures or develop procedures for obtaining the small 
samples of medical records for future validation and audit purposes. 


If cost is a factor for MC+ MCOs calculating performance measures using the Hybrid Method in 
compliance with the HEDIS, SMA and SPHA specifications, then the Administrative Method of 
calculation should be used. 


Ensure that MC+ MCOs understand the need to calculate and report performance measures to 
the SMA and SPHA despite the NCQA requirements and schedule. Given that these measures 
will not always be audited by NCQA auditors due to their rotation schedule, they may be good 
measures to audit in the future. 


MC+ MCOs with significantly lower rates of eligible members and administrative hits should 
closely examine the potential reasons for fewer members or claims identified. This may be due 
to member characteristics, but is more likely due to claims administration procedures and 
system characteristics such as the proportion of members receiving services from capitated 
providers. Identifying methods of improving administrative hits will improve the accuracy in 
calculating the measures. 
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SECTION 4.0 


VALIDATION OF ENCOUNTER DATA 
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4.0 VALIDATION OF ENCOUNTER DATA 





4.1 Definition 


‘For the purposes of this protocol, an encounter refers to the electronic record of a service 
provided to an MCO enrollee by both institutional and practitioner providers (regardless of how 
the provider was paid) when the service would traditionally be a billable service under Fee-for- 


x1 | 


Service (FFS) reimbursement systems. 


An encounter is the unit of service provided to a Member by the MCO. Encounter data provides 
the same type of information found ona claim form. It does not substitute for medical record 
documentation, but should be consistent with and supported by medical record documentation (e.g. 
date of procedure, type of procedure). The MC+ MCOs’ contract with the State Medicaid Agency 
(SMA; Missouri Department of Social Services, Division of Medical Services; DMS) details the 
requirements for an acceptable submission of an encounter. The SMA’s requirements for encounter 


data submitted by the MC+ MCOs include the type of encounter data and required data fields. 


4.2 Purpose and Objectives 


“Encounter data can be used to assess and improve quality, as well as monitor program 
integrity and determine capitation payment rates. However, in order for encounter data to 
effectively serve these purposes, it must be valid; i.e., complete and accurate... This protocol 
specifies processes for assessing the completeness and accuracy of encounter data 
submitted by MCOs and PIHPs to the State. It also can assist in the improvement of the 
processes associated with the collection and submission of encounter data to State 
Medicaid agencies.””!2 


Three objectives for the encounter validation were identified. They included: assessing the quality of 
data for required fields for each claim type; evaluating the representativeness (or completeness) of 
the SMA encounter claims database for MC+ MCO paid and unpaid claims; and validating medical 
records against the SMA encounter claims database. The following were the objectives and 


associated evaluation questions. 


™’ Department of Health and Human Services. Centers for Medicare and Medicaid Services (2002). Validating 
Encounter Data: A protocol for use in Conducting Medicaid External Quality Review Activities, Final Protocol, 
Version 1.0, May 1, 2002. Washington, D.C.: Author. 

12 Ibid. 
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|. The first objective was to obtain a quality baseline of the SMA encounter claim database 
(completeness, accuracy, and reasonableness). The alternative hypothesis was that all data fields 
in the SMA encounter claims database consist of valid (complete, accurate, and reasonable) 
encounter claim data. Appendix 5 shows the recommended minimum criteria established for 
completeness and accuracy of specific data fields. Several evaluation questions were addressed: 


e What is the baseline level of completeness, accuracy, and reasonableness of the critical fields? 
e What is the level of volume and consistency of services? 

e What are the data quality issues associated with the processing of encounter data? 

e What problems are there with how files are compiled and submitted by the MCO? 

e What types of encounter claim data are missing and why? 


2. The second objective was to examine the match between MC+ MCO claims (paid and unpaid) 
and the SMA encounter paid claims database. This would facilitate identification of the level of 
completeness of the SMA encounter claims database as represented by MC+ MCOs paid claims. 
The alternative hypotheses were that 100% of MC+ MCO paid claims are represented in the 
SMA encounter claims database, and 0.00% of MC+ MCO unpaid claims are represented in the 
SMA encounter claims database. Several evaluation questions were posed: 


e What types of paid encounter data are missing and why? 

e What is the fault/match rate of paid and unpaid encounter claims in the SMA encounter claim database and 
the MC+ MCO claims database? 

e What services are being provided that are not being paid? 

e How many services are being provided that are not being paid? 


3. The third objective was to validate the SMA encounter claims (paid) database against medical 
record documentation and obtain a baseline fault (error) rate for the level of accuracy of the 
SMA encounter claims database relative to the services delivered by MC+ MCO providers. The 
alternative hypothesis was that there is a 100% match between the encounter claim data in the 
medical record and the data in the SMA encounter claims database. Accuracy or match rates of 
70% or greater are anticipated for new Medicaid managed care organizations!3. Several 
evaluation questions were addressed: 


e To what extent do the claims in the SMA encounter claims database reflect the information documented in 
the medical record? 

e What is the fault/match rate between SMA encounter claims and medical records? 

e What types of errors are noted? 


3 Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data: 
Second Edition. 
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4.3 Technical Methods 


TIME FRAME 
The dates of service from January |, 2005 through March 31, 2005 were selected by the SMA for 


the three encounter data validation objectives. 


PROCEDURES FOR DATA COLLECTION 

For the first objective, the SMA encounter claims extract file was used to examine the 
completeness, accuracy, and reasonableness of the critical fields and to calculate the rate of each 
claim type per 1,000 members by MC+ MCOs. There are six claim types described in the SMA 
Health Plan Layout Manual: | = Inpatient claim type; M = Medical claim type; O = Outpatient 
Hospital claim type; D = Dental claim type; H = Home Health claim type; and P = Pharmacy claim 
type. Inpatient, Outpatient and Home Health claim types are submitted using a Universal Billing 
(UB-92) file layout, Medical and Dental claim types are submitted using a National Standard 
Format/Centers for Medicare and Medicaid Services 1500 (NSF/CMS 1500) file layout, and the 
Pharmacy claims are submitted using the National Council for Prescription Drug Programs, version 
3 file layout (NCPDP v.3.0). All claims are sent from the MC+ MCOs to the SMA through the SMA 
claims vendor, InfoCrossing, and claim types are assigned by the Medicaid Management Information 


System (MMIS). 


After review and approval of the technical methods and objectives by the SMA, the EQRO 
reviewed, discussed with the SMA, and submitted a data request (see Appendix 6) for the SMA 
encounter claims extract file to be validated for each claim type and each MC+ MCO. The file 
request was made to the SMA on October 27, 2005 and received on December [5, 2005 by the 
EQRO. The SMA reviewed and approved the data request and parameters for the designated the 
fields to be validated by the EQRO. 


For the second objective of comparing the SMA encounter claims with MC+ MCOs’ paid and unpaid 
claims, the SMA encounter claims extract file was parsed by type of file layout (NSF/CMS 1500, UB- 
92, or NCPDP v.3.0) in preparation for matching against MC+ MCO paid and unpaid claims. A 
cross-walk for matching MMIS field names with those of the three national standards file layouts was 
developed and submitted to the SMA for review (February 8, 2005) and approval (March 29, 2005). 
MC+ MCOs were requested to provide paid and unpaid claims for the designated period on the 


sample of members selected by the EQRO. While last year the MC+ MCOs could not consistently 
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provide paid and unpaid claims and/or the requested file layouts which precluded the planned 
analyses, this year internal control numbers (ICNs) were requested to match the paid/unpaid claims 
to the SMA records. However, only two of the seven MC+ MCOs supplied the appropriate 
information required, thus limiting our analyses to these two MC+ MCOs. Additional technical 
assistance is required to improve the data quality required to conduct these analyses in the future. 
The number of Medical encounter claims in the SMA encounter claims extract file was used for 
sample size estimation for the third objective and analysis of the evaluation questions. To examine 
the degree of match between the SMA encounter claims database and medical record procedures 
and diagnoses, 100 encounters from each MC+ MCO were randomly selected from Medical claim 
types for the period of January |, 2005 through March 31, 2005 for medical record review. 
Appendix 7 contains letters of request to providers for medical records, the Table of Contents for 
the Medical Record Review Training Manual, and copies of medical record review tools. Several 


challenges in requesting the data were addressed. 


ANALYSES 

To assess the accuracy and completeness of the SMA encounter claims database, the SMA 
encounter claims extract file for all MC+ MCO paid encounter claims representing services 
rendered from January |, 2005 through March 31, 2005 was analyzed for completeness, accuracy, 
and reasonableness (validity) of the data in each “critical”, or required field examined. The Inpatient, 
Medical, Dental, Home Health, Outpatient Hospital, Pharmacy, and critical fields were chosen by the 


SMA for analysis, with an established threshold of 100% for completion, accuracy, and validity: 
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Medical (NSF/ CMS 1500) Claim Type 
Outpatient Claim Type 

Recipient 1D 

First Date of Service 

Last Date of Service 

Place of Service 

Units of Service 

Procedure Code 

Inpatient Diagnosis (five diagnosis fields) 


Dental (NSF/ CMS 1500) Claim Type 
Outpatient Claim Type 

Recipient 1D 

First Date of Service 

Last Date of Service 

Units of Service 

Procedure Code 


Home Health (UB-92) Claim Type 
Outpatient Claim Type 

Recipient 1D 

First Date of Service 

Last Date of Service 

Units of Service 

Procedure Code 

Revenue Code 

Inpatient Diagnosis (five diagnosis fields) 


Section 4 
Validation of Encounter Data 


Inpatient (UB-92) Claim Type 
Inpatient Claim Type 

Recipient 1D 

Admission Type 

Admission Date 

Discharge Date 

Bill Type 

Patient Discharge Status 
Inpatient Diagnosis (five diagnosis fields) 
First Date of Billing 

Last Date of Billing 

Revenue Code 

Units of Service 


Outpatient Hospital (UB-92) Claim Type 
Outpatient Claim Type 

Recipient 1D 

First Date of Service 

Last Date of Service 

Place of Service 

Units of Service 

Procedure Code 

Inpatient Diagnosis (five diagnosis fields) 


Pharmacy (NCPDP v.3.0) 
Recipient 1D 

Dispensing Date 

Pharmacy Prescription Number 
Drug Quantity Dispensed 
Number of Days Supply 
National Drug Code 


Each field was examined for the presence or absence of data (completeness), the correct type and 
size of information (accuracy), and the presence of valid values (reasonableness) or validity using the 
criteria listed below. Appendix 5 contains the parameters for the validation of encounter claims 
fields for each of the six encounter claim types, the procedure codes, and the diagnosis codes. 


Appendix 5 also shows the recommended threshold for validity of specific data fields. 


Completeness: The extent to which an encounter claim field contains data (either 
present or absent). 


Accuracy: The extent to which an encounter claim field contains the correct 
type of information (e.g., numeric, alpha, alphanumeric) in the 
proper format (e.g., mm/dd/yyyy for date field). 


Reasonableness (Validity): The extent to which an encounter claim field represents a valid value 
(e.g., an actual procedure code, actual birth date) 





For the validation of the SMA encounter claims extract file with MC+ MCO medical records, the 
goal was to validate the procedure code and diagnosis code fields in the SMA encounter claims 
database against the information provided in the medical record. The minimum number of records 


required for the evaluation of two variables (procedure and diagnosis) with an estimated error rate 


Performance Management Solutions Group 128 


etormmnaace Management 
Huthens Group 
5 Coe ol are 


A ese ae A division of Behavioral Health Concepts, Inc. 


Missouri MC+ Managed Care External Quality Review Section 4 
Report of Findings — 2005 Validation of Encounter Data 


of 30% (based on Medstat estimates'!*), reliability of 1.96 (95% statistical significance), and a 
meaningful difference of 55% were calculated using the number of Medical encounters in the SMA 
encounter claims file for each MC+ MCO (see Equation |). There were no differences in the 
number of required records for MC+ MCOs, with the minimum required sample size of 88. A total 
of 100 encounters for each MC+ MCO were randomly selected for medical record review using a 


probability sample. 


Figure 14 - Formula for Calculating Minimum Required Sample Size 
2 - 
_ z2NP,(1-P,) 
(N -1) 2P,2 + z2P,(1 - P,) 


Where Py, = Estimated True Error Rate; meaningful difference between true and estimated value ; z = level of reliability; 
€=|(Py - meaningful difference)/meaningful difference; N = number of Medicaid Claim Types for the period January |, 
2004-March 31, 2004; n = Minimum required sample size!> 


4.4 Findings 


One limitation of the present analysis is that the encounter claim completeness and accuracy analysis 
was based on paid encounter claims and did not account for all claims that were submitted and 
rejected through system edits. Also, because the SMA encounter claims extract file was for service 
dates from January |, 2005 through March 31, 2005, some service dates might extend beyond this 
period. For example, if the first date of service was later in the period (e.g., March 31, 2005), the last 
date of service may extend beyond the period specified by SMA parameters for the validation 
process (e.g., a Discharge Date of April |, 2005). When last dates of service appeared to be within 
a reasonable period, dates outside the valid range were not interpreted and are presented for 
informational purposes only. In addition, the second through fifth diagnosis code fields are required 
when the information is available. Not all encounters had five diagnoses. Therefore, |00.00% 
completion of these fields would not be expected. Conclusions regarding the extent to which the 
encounter claims database reflects the accuracy and completeness of rejected claims cannot be 
drawn. Data are presented in the aggregate and are available at the MC+ MCO level in the 
individual MC+ MCO summaries. The findings of the encounter data validation are presented in 


response to each evaluation question, by claim type and critical field for all MC+ MCOs. 


™ Medstat (1999). A Guide for States to Assist in the Collection and Analysis of Medicaid Managed Care Data: 
Second Edition. 
™® Levy, P.S. & Lemeshow, S. L. (1999). Sampling of Populations: Methods and Applications, Third Edition, 
John Wiley and Sons: New York; see box 3.5 for Exact and approximate sample sizes required under simple 
random sampling for proportions. 
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What is the Baseline Level of Completeness, Accuracy, and Reasonableness of 
the Critical fields? What Types of Encounter Claim Data are Missing and Why? 


For the Medical claim type, there were a total of 997,736 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005 (see Table 40). 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 


3. The Outpatient First Date of Service field was 100.00% complete and accurate, and 99.58% valid. Invalid dates of 
service ranged from 01/04/2004 — 12/31/2004. 


4. The Outpatient Last Date of Service field was 100.00% complete and accurate, and 99.87% valid. Invalid dates of 
service ranged from 04/01/2005 — 10/03/2005. 


The Outpatient Units of Service field was 100.00% complete, accurate, and valid. 
The Outpatient Procedure Code field was |00.00% complete, accurate, and valid. 


The Outpatient Place of Service field was 100.00% complete, accurate and valid. 


eS S&S 


The first Diagnosis Code field was 99.999% complete, accurate and valid. The remaining fields were left blank 
(incomplete, inaccurate, and invalid). 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health Plan Record 
Layout Manual, they fell well below the 100% threshold established by the SMA for this validation. The second 
Diagnosis Code field was 18.17% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The third Diagnosis Code field was 2.38% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 0.73% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The fifth Diagnosis Code field was 0.00% complete, accurate and valid. All fields were 
blank (incomplete, inaccurate, and invalid). 


ae. Performance Management Solutions Group 130 


oe ran BE A division of Behavioral Health Concepts, Inc. 





Missouri MC+ Managed Care External Quality Review Section 4 
Report of Findings — 2005 Validation of Encounter Data 


Table 40 - Encounter Data Validation of Critical Fields, All MC+ MCOs, Medical Claim Type 


Correct Type of 
Mahelaaitchatelam da-i-)0h Oo) a a -Vel a) 4) Mohiclacirehatele | WEVICoMmYE- TIO 


Critical or Fatal Field # % | # : % # % : # | % 


Recipient |D [OUTPAT-PROCESSED-RECIP-ID] 997,736 100.00% 997,736 100.00% 997,736 100.00% 997,736 100.00% 


Last Date of Service [OUTPAT-LAST-DT-SVC] 997,736 100.00% 997,736 100.00% 997,736 100.00% 996,453 99.87% 


Diagnosis S{OUTPAT-OX) ig .00% | 0.00% | | 0.00% 00% 


Total Claims 999,140 





Note: Based on state extract file of dates of service from January |, 2005 - March 31, 2005. 
Source: Missouri Department of Social Services, Division of Medical Services, December 15, 2005. 
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For the Dental claim type, there were | 16,148 encounter claims paid by the SMA for the period 


January |, 2005 through March 31, 2005. All fields examined were |00.00% complete, accurate and 
valid for all MC+ MCOs. 


For the Home Health claim type, there were a total of 1,192 encounter claims paid by the SMA for 


the period January |, 2005 through March 31, 2005. 


2 
3 
4. 
5 
6 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete, accurate and valid. 
The Last Date of Service field was 100.00% complete, accurate and valid. 
The Units of Service field was 100.00% complete, accurate and valid. 


The Procedure Code field was 73.83% complete and accurate (i.e., 312 fields left blank), and 70.64% valid. Invalid 
fields were blank or contained entries of “99601” (n = 5) or “Y9009” (n = 33). 


For the Inpatient claim type, there were a total of 94,675 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005 (see Table 41). 


2 
3. 
4 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 
The Admission Type field was 100.00% complete, accurate, and valid. 


The Admission Date field was 100.00% complete, accurate, and 94.96% valid. Invalid dates ranged from 
02/24/2004 — 12/31/2004. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). The correct type 
of information (date format) was present 98.15% (with |747 entries of “99999999”). Valid values were present 
93.82% of the time. In addition to the invalid “99999999” entries, invalid dates ranged from 04/01/2005 — 
05/23/2005. 


The Bill Type field was 100.00% complete, accurate and valid. 


The Patient Status field was 100.00% complete with the correct number of characters (size) and accurate, and 
99.95% valid. Invalid fields contained codes “00”, “09”, “61”, and “63”, which were the correct type but not 
listed under the valid codes list. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


The second, third, fourth, and fifth Diagnosis Code fields fell below the 100% threshold for completeness, 
accuracy, and validity established by the SMA (95.03%, 86.03%, 70.74%, and 51.93%, respectively). The remaining 
fields were blank (incomplete, inaccurate, and invalid). 


The First Date of Service field was 100.00% complete, accurate, and 95.61% valid. Invalid dates of service ranged 
from 10/10/2004 — 01 2/31/2004. 


. The Last Date of Service field was 100.00% complete and accurate, and 95.35% valid. Invalid dates of service 


ranged from 04/01/2005 — 05/23/2005. 


The Revenue Code field was 99.92% complete, accurate, and valid. The Units of Service field was 100.00% 
complete, accurate and valid. 
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Table 41 - Encounter Data Validation of Critical Fields, All MC+ MCOs, Inpatient Claim Type 





Critical or Fatal Field 
Inpatient Claim Type [INPAT-CLAIM-TYPE] 
Recipient 1D [INPAT-RECIP-ID] 
Admission Type [INPAT-ADMIT-TYPE] 
Admission Date [INPAT-ADMIT-DT] 
Discharge Date [INPAT-MED-DSCHG- DT] 
Inpatient Bill Type [INPAT-BILL-TYPE] 
Patient Status [INPAT-PATIENT-STAT] 
Diagnosis [INPAT-DX] 
Diagnosis [I NPAT-DX] 
Diagnosis [INPAT-DX] 
Diagnosis [I NPAT-DX] 
Diagnosis [INPAT-DX] 
First Date of Service [INPAT-FIRST-DT-SVC] 
Last Date of Service [INPAT-LAST-DT-SVC] 
Revenue Code [INPAT-REVENUE-CD] 
Units of Service [INPAT-UNITS-SV] 





Total Claims 


Information Present 


# 
94,675 
94,675 
94,675 
94,675 
94,675 
94,675 
94,675 
94,675 
89,973 
81,451 
66,973 
49,161 
94,675 
94,675 
94,602 
94,675 


% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 

95.03% 
86.03% 
70.74% 
51.93% 
100.00% 
100.00% 
99.92% 
100.00% 


Correct Size 


# 
94,675 
94,675 
94,675 
94,675 
94,675 
94,675 
94,675 
94,675 
89,973 
81,451 
66,973 
49,161 
94,675 
94,675 
94,602 
94,675 


Note: Based on state extract file of dates of service from January |, 2005- March 31, 2005. 
Source: Missouri Department of Social Services, Division of Medical Services, December [5, 2005 
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% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 
100.00% 

95.03% 
86.03% 
70.74% 
51.93% 
100.00% 
100.00% 
99.92% 
100.00% 


Validation of Encounter Data 


Correct Type of 
Information 


# 
94,675 
94,675 
94,675 
94,675 
92,928 
94,675 
94,675 
94,675 
89,973 
81,451 
66,973 
49,161 
94,675 
94,675 
94,602 
94,675 


% 
100.00% 
100.00% 
100.00% 
100.00% 
98.15% 
100.00% 
100.00% 
100.00% 
95.03% 
86.03% 
70.74% 
51.93% 
100.00% 
100.00% 
99.92% 
100.00% 


Valid Value 


# 
94,675 
94,675 
94,675 
89,908 
88,821 
94,675 
94,630 
94,675 
89,973 
81,451 
66,973 
49,161 
90,515 
90,274 
94,602 
94,675 


% 
100.00% 
100.00% 
100.00% 
94.96% 
93.82% 
100.00% 
99.95% 
100.00% 
95.03% 
86.03% 
70.74% 
51.93% 
95.61% 
Sooo ve 
99.92% 
100.00% 
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For the Outpatient Hospital claim type, there were a total of 601,493 encounter claims paid by the 
SMA for the period January |, 2005 through March 31, 2005 (see Table 42). 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 


The First Date of Service field was 100.00% complete and accurate, and valid. 


2 
3 
4. The Last Date of Service field was 100.00% complete and accurate, and valid. 
5. The Units of Service field was 100.00% complete, accurate and valid. 

6 


The Outpatient Procedure Code field was 94.00% complete. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The fields were 93.97% accurate and 92.92% valid. There were a small number of fields 
with procedure codes less than five alphanumeric characters. A large proportion of the invalid codes were 
outside the specified codes (i.e., 80048-89399) when the Revenue Code ranged 300-319. 


7. The Outpatient Revenue Code field was 99.96% complete and accurate and 91.83% valid. Invalid codes were 
primarily accounted for by invalid entries of “000” from HealthCare USA. 


8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the Health Plan Record 
Layout Manual, they fell well below the 100% threshold established by the SMA for this validation. The second 
Diagnosis Code field was 67.24% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The third Diagnosis Code field was 45.95% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 23.02% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). The fifth Diagnosis Code field was 10.51% complete, accurate and valid. The remaining 
fields were blank (incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were 772,269 claims paid by the SMA for the period January I, 
2005 through March 31, 2005. All fields examined were 100.00% complete, accurate and valid 
(Recipient ID, First Date of Service, Prescription Number, Quantity Dispensed, Days Supply, and 
National Drug Code). 
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Table 42 - Encounter Data Validation of Critical Fields, All MC+MCOs, Outpatient Hospital Claim Type 


Correct Type of 
Mabcelast-haleleM a a--1-)0h Correct Size | I ———a — Value 


_ Critical or Fatal Field # re | | # | % | % aw eae 


Outpatient Outpatient Claim Type [OUTPAT-CLAI M- TYPE] tts 601,493 493 100. 00% 601,493 100.00% 601,49: 493 100. 00% 601,49: 493 100. 00% 


First Date of Service [OUTPAT-FIRST-DT-SVC] 601,493, 100.00% 601,493 100.00% 601,493, 100.00% 601,493 100.00% 
Units of Service [OUTPAT-UNITS-SVC] 601,493 100.00% 601,493 100.00% 601,493 100.00% 601,493 100.00% 


Revenue Code [OUTPAT-REVENUE-CODE] 601,255 99.96% 601,255 99.96% 601,255 99.96% 552,377 91.83% 
Diagnosis [OUTPAT-DTL-DIAG- CODE] 404,296 67.24% 404,296 67.24% 404,296 67.24% 404,296 67.24% 
Diagnosis [OUTPAT-DTL-DIAG- CODE] 138,490 23.02% 138,490 23.02% 138,490 23.02% 138,490 23.02% 


Total Claims 601,493 





Note: Based on state extract file of dates of service from January |, 2005- March 31, 2005. 
Source: Missouri Department of Social Services, Division of Medical Services, December 15, 2005. 
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What is the Level of Volume and Consistency of Services? 

One method of examining the level, consistency, and volume of services is to assess the extent to 
which each MC+ MCO is consistent with the remaining MC+ MCOs and the average of all MC+ 
MCOs services represented in the SMA encounter claims database. The level, consistency, and 
volume of services represented in the SMA encounter claims database is a function of the 
acceptance of encounter claim submissions. It is also a function of the process of manipulation of 
data from national standard layouts for Medical (NSF/CMS 1500); Dental (NSF/CMS 1500); Inpatient, 
Outpatient Hospital, Home Health (UB-92); and Pharmacy claims (NCPDP 3.0) into the State MMIS 
system edits. Additionally, the entry and transmission of data by MC+ MCQOs, vendors, and 
providers, the accessibility of services, member utilization patterns, and provider practice patterns 
influence the data. Given the data issues experienced by the EQRO during the last audit, we feel 
that the results of this year (2005) data should be used as a baseline for future analysis and 
interpretation by the SMA and the MC+ MCOs. With the large number of members enrolled in 
each MC+ MCO, it was expected that factors such as physician practice patterns and member 
utilization patterns would not have a statistically significant impact on the findings, resulting in all 
MC+ MCOs having similar rates of encounters per 1,000 members as the rate for all MC+ MCOs. 
Statistically significant findings are more likely a function of the data quality and completeness 
resulting from the processing of data by providers, vendors, MC+ MCOs, and the MMIS rather than 


the accessibility or quality of services. 


Using the SMA encounter claims extract file from January |, 2005 through March 31, 2005, the 
volume of services for each claim type and MC+ MCO was examined. The rate of each claim type, 
regardless of the accuracy, consistency, and validity of the data was examined. The rate of claims 
per 1,000 members based on one quarter of data was calculated by dividing the number of members 
enrolled as of the last week of March 2005, by 4, then calculating the rate of claims per |,000 
members. Figures |4 through 19 illustrate the rates of claim types and the results of two-tailed z- 
tests comparing each MC+ MCO with the statewide rate of claims. Statistically significant 
differences between an MC+ MCO and the rate for all MC+ MCOs at the 95% level of statistical 
significance are indicated by an asterisk. The 95% upper and lower confidence limits are 
represented by the black bars on the y-axis. For comparisons that were statistically significant at the 
95% confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), and the 
significance levels (p < .05) are reported. When there was no statistical significance, the significance 


level is reported as “not significant” (n.s.). 
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Medical encounter claim types consist of claims submitted by providers, vendors, and MC+ MCOs. 
As shown in Figure |5, there was some variability across MC+ MCOs in the statewide rate per 
|,000 members of Medical encounter claim types compared to the rate for all MC+ MCOs 
(9,031.78 Medical encounter claims per 1,000 members). One MC+ MCO (Missouri Care, 

| 1480.89, z = 1.62; 95% Cl: 10373.71, 12588.07; p < .01) showed a significantly higher rate, while 
one MC+ MCO (Community CarePlus, 6693.60, z = -1.58; 95% Cl: 5586.42, 7800.78; p < .05) hada 


significantly lower rate of Medical encounter claims than the rate for all MC+ MCOs. 


Figure 15 - Medical Encounters Claim Types per 1,000 Members, January |, 2005 — March 31, 2005 
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CCP MHP HCUSA MOCare FHP FG BA+ All MC+ 
MCOs 
MC+ MCO 
6,693.60 8,936.47 9,208.91 11,480.89 9,171.73 7,973.26 9,903.17 9,031.78 


Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2005 — March 31, 2005 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2005 was used to calculate the rate per 1,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 


Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, 
enrollment for all Waivers. 
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Dental encounter claims consist of claims submitted by providers, vendors, and MC+ MCOs. As 
shown in Figure |6, there was some variability across MC+ MCOs in the rate per 1,000 members of 
Dental encounter claims compared to the rate for all MC+ MCOs (1051.40 Dental encounter claims 
per 1,000 members). Two MC+ MCOs (HealthCare USA, 1305.12, z = 1.01; 95% Cl: 1019.81, 
1590.42; p < .05; and Family Health Partners, |253.10, z = .87; 95% Cl: 967.80, 1538.40; p < .05) 
had significantly higher rates. While two MC+ MCOs (Community CarePlus, 453.88, z = -1.20; 95% 
Cl: 168.58, 739.19; p < .05; and Missouri Care, 327.42, z = -1.53; 95% Cl: 42.12, 612.72; p < .05) 
had significantly lower rates of Dental encounter claims per |,000 members than the rate for all 


MC+ MCOs. 


Figure |6 - Dental Encounters per | ,000 Members, January |, 2005 — March 31, 2005 
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CCP MHP HCUSA MOCare FHP FG BA+ All MC+ 
MCOs 
MC+ MCO 
453.88 986.15 1,305.12 327.42 1,253.10 937.53 1,159.97 1,051.40 


Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2005 — March 31, 2005 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2005 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 

Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, 
enrollment for all Waivers. 
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There were very few Home Health encounter claim types submitted by MC+ MCOs. Two MCt+ 
MCO (FirstGuard, 54.51, z = 1.43; 95% Cl: 35.00, 74.03; p < .0| and Blue-Advantage Plus of Kansas 
City, 56.04, z = 1.49; 95% Cl: 36.53, 75.55; p < .01) submitted significantly higher rates of Home 


Health encounter claims than the rate for all MC+ MCOs (10.79; see Figure | 7.) 


Figure 17 - Home Health Encounter Claim Types per 1,000 Members, January I, 2005 — March 31, 2005 
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Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2005 — March 31, 2005 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2005 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 


Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, 
enrollment for all Waivers. 
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Inpatient encounter claim types consist of claims submitted by hospital facilities and MC+ MCOs. 
As shown in Figure 18, there was some variability across MC+ MCOQOs in the rate per |,000 
members of Inpatient encounter claims compared to the rate for all MC+ MCOs (857.02 Inpatient 
encounter claims per 1,000 members). Three MC+ MCOs had significantly higher rates of Inpatient 
encounter claims (Health Care USA, | 153.03, z = .89; 95% Cl: 782.77, 1523.29, p < .05; Missouri 
Care, 1324.80, z = 1.23; 95% Cl: 954.54, 1695.06; p < .05; Family Health Partners, | 193.65, z = .97; 
95% Cl: 823.39, 1563.91; p < .05). Two MC+ MCOs had significantly lower rates of Inpatient 
encounter claims (Community CarePlus, 195.89, z = -1.03; 95% Cl: 0, 566.15; p < .01; Blue- 
Advantage Plus of Kansas City, 144.43, z = -1.32; 95% Cl: 0, 514.69; p < .O1) than the rate for all 
MC+ MCOs. 


Figure |8 - Inpatient Encounter Claim Types per 1,000 Members, January |, 2005 — March 31, 2005 
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195.89 504.39 1,153.03 1,324.80 1,193.65 456.62 144.43 857.02 


Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per 1,000 
members = Number Claims January |-2005 — March 31, 2005 / (Number members / 4) X 1,000. Enrollment as of the 
last week of March 2005 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 

Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, 
enrollment for all Waivers. 
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Outpatient Hospital encounter claim types consist of claims submitted by outpatient hospital 
facilities and MC+ MCOs. As shown in Figure 19, there was some variability across MC+ MCOs 
compared to the rate for all MC+ MCOs (5444.88 Outpatient Hospital encounter claims per |,000 
members). Two MC+ MCOs had significantly higher rates of Inpatient encounter claims (Missouri 
Care, 7859.08, z = 1.44; 95% Cl: 6667.85, 9050.32; p < .01; Family Health Partners, 7384.80, z = 
1.15; 95% Cl: 6193.57, 8576.04; p < .O1). While two MC+ MCOs had significantly lower rates of 
Outpatient Hospital encounter claims per |,000 members (Community CarePlus, 3976.98, z = -.97; 
95% Cl: 2785.75, 5168.21; p < .05; and Blue-Advantage Plus of Kansas City, 3809.46, z = -1.07; 95% 
Cl: 2618.23, 5000.76; p < .05) than the rate for all MC+ MCOs. 


Figure 19 - Outpatient Hospital Encounter Claim Types per 1,000 Members, January |, 2005 — March 31, 
2005 
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Cor MHP HCUSA MOCare FHP FG BA+ All MC+ 
MCOs 
MC+ MCO 
3,976.98 5,946.62 5,197.24 7,859.08 7,384.80 4,533.53 3,809.46 5,444.88 


Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2005 — March 31, 2005 / (Number members / 4) X 1,000. Enrollment as of the 
last week of March 2005 was used to calculate the rate per 1,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 

Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, 
enrollment for all Waivers. 
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Pharmacy encounter claim types consist of claims submitted by pharmacy providers and MC+ 
MCOs. As shown in Figure 20, there was some variability across MC+ MCOs in the statewide rate 
per 1,000 members of Pharmacy encounter claims compared to the rate for all MC+ MCOs 
(6990.79 Pharmacy encounter claims per |,000 members). One MC+ MCO (Missouri Care, 
8877.67, z = 1.87, 95% Cl: 8100.94, 9654.40; p < .05) had a significantly higher rate of Pharmacy 
encounter claims. While two MC+ MCO (Mercy Health Plan, 5605.56, z = -1.25; 95% Cl: 4828.48, 
6382.29; p < .0|; and Blue-Advantage Plus of Kansas City, 6039.98, z = -.84; 95% Cl: 5263.25, 


6816.71; p < .05) had a significantly lower rate of Pharmacy encounter claims per |,000 members 


than the rate for all MC+ MCOs. 


Figure 20 - Pharmacy Encounter Claim Types per |,000 Members, January |, 2005 — March 31, 2005 
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6,820.68 5,605.56 7,213.51 8,877.67 7,226.18 6,652.62 6,039.98 6,990.79 


Note: Rate per 1,000 members is an estimated annual rate based on first quarter state encounter data; Rate per |,000 
members = Number Claims January |-2005 — March 31, 2005 / (Number members / 4) X 1,000. Enrollment as of the last 
week of March 2005 was used to calculate the rate per |,000 members. Error bars on the y-axis represent 95% 
confidence intervals; * Indicates values are significant at the 95% level of significance, two-tailed z-test. 

Source: Missouri Department of Social Services, Division of Medical Services, State Session MPRI screen, 
enrollment for all Waivers. 
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Table 43 and Figure 21 show the proportion of claim types for each MC+ MCO based on the SMA 
encounter claims extract file. Community CarePlus had the highest proportion of Pharmacy claims 
relative to all other MC+ MCOs; Blue-Advantage Plus of Kansas City had the highest proportion of 


the Dental and Medical claim types; Children’s Mercy Family Health Partners had the highest 


proportion of hospital claims; and HealthCare USA had the highest proportion of Inpatient claims. 


There were no patterns observed across MC+ Plans, suggesting that the variations are not related 


to member or provider practice characteristics. 


Table 43 - 


Numerical Proportion of Claim Types per MC+ MCO, January I, 2005 —March 31, 2005 


Home 
Health Total 


6820.68 453.88 3976.98 0.09 6693.60 195.89 18141.12 
MHP 5605.56 986.15 5946.62 2.47 8936.47 504.39 21981.66 
HCUSA 7213.51 1305.12 5197.24 1.38 9208.91 1153.03 24079.19 
MOCare 8877.67 327.42 7859.08 0.00 11480.89 1324.80 29869.86 
FHP 7226.18 1253.10 7384.80 2.81 9171.73 1193.65 26232.27 
FG 6652.62 Sie). CieeiSs 5) 54.51 7973.26 456.62 20608.07 
BA+ 6039.98 1159.97 3809.46 56.04 9903.17 144.43 21113.06 


~ 9031.78 | 857.02 23386.67 





AllMC+ MCOs | 6990.79 


1051.40 5444.88 10.79 


Figure 21 - Percentage Proportion of Claim Types per MC+ MCO, January |, 2005 — March 31, 2005 
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Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, 
December [5, 2005. 
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To What Extent do the MC+ MCO claims (paid and unpaid) match the State 
Encounter Claims Paid Claims Data Base? 

Only two (Missouri Care and Health Care USA) of the seven MC+ MCOs submitted the requested 
internal control numbers (ICNs) generated by the SMA data system. All encounter claims submitted 
by Missouri Care were of “paid” claim status. Health Care USA submitted encounter claims that 
were “paid” or “denied” status, however no “unpaid” claims were submitted by any of the MC+ 


MCOs. 


The ICNs were used to match the encounters of each claim type (Inpatient, Outpatient, and 
Pharmacy) between the MC+ MCO and the SMA extract files. A “match” was considered if the 


MC+ MCO sample encounter was identified in the SMA database. 


What types of paid encounter data are missing and why? 


There were unmatched “paid” encounters within all claim types (Inpatient, Outpatient, and 


Pharmacy) examined for Missouri Care and Health Care USA health plans. 


For HealthCare USA, the majority of unmatched encounters were due to missing ICN numbers, 
which are required to match the encounter to that of the SMA. Within the Pharmacy Claim type, 
66.6/7%of the thirty-three unmatched encounters were missing ICN numbers. Therefore, only | | 
Pharmacy claims were legitimately missing from the SMA data. For the Outpatient data, 96.88% of 
the 192 unmatched claims were missing ICNs. Of the six unmatched claims with ICNs, two of 
those were of “denied” status and would not be expected to be present in the SMA file. Thus, only 
four unmatched encounters were legitimately missing from the SMA data records. For Inpatient 


Claims, all unmatched claims were missing ICNs. 


For Missouri Care, a majority of unmatched encounters were due to missing ICN numbers, which 
are required to match the encounter to that of the SMA. Within the Pharmacy Claim type, 34.29% 
of the thirty-five unmatched encounters were missing ICN numbers. Therefore, 23 Pharmacy 
encounter claims were legitimately missing from the SMA extract data. For the Outpatient data, 
85.58% of the 214 unmatched claims were missing ICNs. Thus, there were 33 unmatched 
encounters that were legitimately missing from the SMA data records. For Inpatient Claims, all 


unmatched claims were missing ICNs. 
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What is the fault/match rate of paid and unpaid encounter claims in the SMA 
encounter claim database and the MC+ MCO claims database? 


Since only HealthCare USA and Missouri Care included internal control numbers that matched 
those of the SMA, BHC only conducted the planned analysis of comparing MC+ MCO encounter 


data to the SMA encounter claim extract file for these two MC+ MCOs. 


For Health Care USA, all of the Pharmacy Claim type data submitted to the EQRO (n = 333,870) 
was of “paid” status. There were a total of 33 unmatched claims that were in the HCUSA 
encounter file and absent from the SMA data. Thus, 99.99% of the HCUSA submitted encounters 
matched with the SMA encounter records. For all Outpatient Claim Types (Medical, Dental, Home 
Health, & Hospital; n = 727,244), 79 “denied” claims were submitted by HCUSA but all other 
encounter claims were of “paid” status. Of the encounter claims submitted by HCUSA, a total of 
|92 records were unmatched with the SMA encounter data. There was a “hit” rate of 99.98% 
between HCUSA encounter claims and the SMA encounter data. For the Inpatient Claim Type, 
HCUSA submitted 53,367 encounter claims. Only 23 of these encounter claims were of “denied”’ 
status; all other claims were of “paid” status. There were a total of 40 unmatched records between 


HCUSA and the SMA, yielding a 99.93% “hit” rate. 


For Missouri Care, all of the Pharmacy claims submitted to the EQRO (n = 78,685) were of “paid” 
status. There were a total of 35 unmatched claims that were in the MOCare encounter file and 
absent from the SMA data. Thus, 99.96% of the EQRO submitted encounters matched with the SMA 
encounter records. For all Outpatient Claim Types (Medical, Dental, and Hospital), MOCare 
submitted | 74,317 “paid” encounters. Of these encounter claims a total of 214 records did not 
match with the SMA encounter claim extract file. There was a “hit” rate of 99.88% between 
MOCare encounter claims and the SMA encounter data. For the Inpatient Claim Type, MOCare 
submitted | 1,742 encounter claims. All encounter claims were of “paid” status. Seventy-nine (79) 


encounters from MOCare did not match the SMA extract file, yielding a 99.33% “hit” rate. 


What services are being provided that are not being paid and how many services are 
being provided that are not being paid? 


There were no “unpaid” encounter claims submitted to the EQRO by an MC+ MCO. 
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To What Extent Do the Claims in the State Encounter Claims Database Reflect the 
Information Documented in the Medical Record? What is the Fault/Match Rate 
between State Encounter Claims and Medical Records? 


Table 44 shows the population (number of encounters), minimum required sample size, the number 
of encounters sampled, and the number and rate of records submitted for review. Of the 997,736 
Medical encounter claim types in the SMA encounter claims extract file for January |, 2005 through 
March 31, 2005, 700 encounters (100 per MC+ MCO) were randomly selected. This was an 
oversample, as the minimum required sample size was 88 per MC+ MCO. Providers were 
requested to submit medical records for review. For the /00 selected encounters, there were 607 
medical records (86.71%) submitted for review. MC+ MCO submission rates ranged from 76.0% 
(Mercy Health Plan and First Guard) to 100.0% (Missouri Care). Encounters for which no 
documentation was submitted were unable to be validated. Table 45 and Figure 22 show the results 
of the match for procedures. Across all MC+ MCOs, 52.0% of the medical records contained 
matching procedure codes or descriptors. MC+ MCO match rates ranged from 42.0% (HealthCare 
USA and Community CarePlus) to 71.0% (Missouri Care). Two MC+ MCOs (Missouri Care, 71.0%; 
z= 1.63, 95% Cl: 62.4, 79.6; p < .O1; and Blue-Advantage Plus of Kansas City, 65.0%, z =I.1 1, 95% 
Cl: 56.4, 73.6; p < .O1) had match rates significantly higher than the rate for all MC+ MCOs. While 
three MC+ MCOs ( HealthCare USA, 42.0%, z = -.86, 95% Cl: 33.4, 50.6, p < .05; Mercy Health 
Plan, 43.0%, z= -.77, 95% Cl: 34.4, 51.6, p < .05; and Community CarePlus, 42.0%, z = -.86, 95% Cl: 
33.4, 50.6, p < .05) had significantly lower rates. The CMS Protocols suggest a 99% match rate as a 
validity criterion. The fault rate for all MC+ MCOs for the procedure was 48.0%, with MC+ MCO 
fault rates ranging from 29.0% to 58.0%. When considering only the documentation submitted for 
review, the match rate for all MC+ MCOs for procedures was 59.97%, while the match rate for 


diagnoses was 99.01%. 
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Table 44 - Encounter Data Validation Samples and Medical Record Submission Rate 
Number 
Number My C=te | Cer] 
Number Minimum Encounters AX =Tote) ge fj Submission 
MC+ MCO Encounters | Sample Size Sampled Received Rate 


Community Care Plus 77,055 
Mercy Health Plan 105,046 


Health Care USA 426,225 
Missouri Care 101,758 
Family Health Partners 117,554 
FirstGuard 84,391 


Blue Advantage Plus 
All MC+ MCOs 86.71% 


Note: The number of encounters represents the number of unique Medical claim types found in the SMA encounter claims 
extract file for the period January |, 2005 through March 31, 2005. The minimum sample size is based on the validation of 
medical records for two dependent variables, the procedure code and the diagnosis code. Number Encounters Sampled = 
Number of unique encounters randomly sampled by the EQRO for medical record review validation. Number Medical 
Records Received = Number medical records submitted by MC+ MCO providers; Number Claim Forms Received = 
Number claim forms submitted by MC+ MCO providers; Submission Rate = Proportion of medical records submitted of 
the number of encounters sampled. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, 
December |5, 2005. BHC, Inc. 2005 External Quality Review Validation of Encounter Data. 
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Table 45 - Procedure Validation Rate 





Section 4 


Validation of Encounter Data 





Number Rate Validated aa 
Number Medical ro} mm (=Ke ier J Actual 
Encounters LX =Tere} me f-) Number LX =Tere} we f-) WEViTerhatela 
| Oh a | OL] Sampled Received | Validated Received Rate oe & UCL 

Community Care Plus 100 77 42 54.55% 42.00% 58.00% 33.36% 50.64% 
Mercy Health Plan 100 76 43 56.58% 43.00% 57.00% 34.36% 51.64% 
Health Care USA 100 91 42 46.15% 42.00% 58.00% 33.36% 50.64% 
Missouri Care 100 100 71 71.00% 71.00% 29.00% 62.36% 79.64% 
Family Health Partners 100 98 50 51.02% 50.00% 50.00% 41.36% 58.64% 
FirstGuard 100 76 51 67.11% 51.00% 49.00% 42.36% 59.64% 
73.03% 65.00% 35.00% 56.36% 73.64% 


Blue Advantage Plus 
All MC+ MCOs 


ee 59.97%| __52.00%| _48.00%| _43.36% 60.64% 


Note: Number Encounters Sampled = Number of unique encounters randomly sampled by the EQRO for medical record review validation; Number Medical Records 
Received = Number medical records submitted by MC+ MCO providers for validation; Number Validated = Number of encounters for which there was a similar or 
matching procedure code or description on the claim form, or adequate documentation in the medical record to support the procedure code as judged by a professional 
medical coder. Rate Validated of Medical Records Received = Number Validated/Number Medical Records Received; Actual Rate Validated = Number Validated/Number 
Encounters Sampled; LCL = Lower Confidence Limit at the 95% confidence level; UCL = Upper Confidence Limit at the 95% confidence level. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file December [5, 2005. BHC, Inc. 2005 External 


Quality Review Validation of Encounter Data. 
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Figure 22 - Encounter Data Procedure Validation Rate, January |, 2005 — March 31, 2005 
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Note: * Indicates values are significant at the 95% level of significance, two-tailed z-test. See corresponding tables for 95% 
confidence intervals. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, 
December 15, 2005. BHC, Inc. 2005 External Quality Review Validation of Encounter Data. 


For the validation of the diagnosis, 60! (85.86%) matched the diagnosis found in the SMA encounter 
claims extract file across all MC+ MCOs (see Table 46 and Figure 23). MC+ MCO match rates 
ranged from 71.0% (Community CarePlus) to 100.0% (Missouri Care) of the medical records or 
claim forms for diagnosis codes. Two MC+ MCOs (Missouri Care, 100.0%, z =.98, 95% Cl: 90.3, 
100.0; p < .OI; and Family Health Partners, 98%, z =.83, 95% Cl: 88.3, 100.0; p < .05) had match 
rates significantly higher than the rate for all MC+ MCOs; while three MC+ MCOs (Mercy Health 
Plan, 76.0%, z = - .86, Cl: 66.3, 85.7; p < .05; Community CarePlus, 71.0%, z = - 1.24, Cl: 61.3, 80.7; 
p< .Ol; and First Guard, 76.0%, z = - .86, Cl: 66.3, 85.7; p < .05) had significantly lower rates. The 
CMS Protocol suggests a greater than 90% validity criterion.'¢ The fault rate for all MC+ MCOs on 


the diagnosis was 14.14%, with MC+ MCO fault rates ranging from 0% to 29.0%. 


*© Validating Encounter Data, A protocol for use in Conducting Medicaid External Quality Review Activities, 
Department of Health and Human Services, Centers for Medicare and Medicaid Services, Final Protocol, Version 
1.0, May 1, 2002. 
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Table 46 - Diagnosis Validation Rate 
Number ! 
Number (axe | fore] | Rate Validated of) Actual 
Salecolelaha-l as |X =Yere] we f=} Number MY (=Xe | for-] | Records | WE VIC Latele Saaelam@ar-1ei ie) 
MC+ MCO Requested Received WEVICe Fh k=re| Received Rate Rate 


Community CarePus | 300) el yaGon ie 


Health Care USA ee 100.00% 91.00% 9.00% 81.32%  100.00% 
Family Health Partners ee 100.00% 98.00% 2.00% 88.32%  100.00% 
Blue Advantage Plus a 100.00% 89.00% 11.00% 79.32% 98.68% 


All MC+ MCOs 99.01% 85.86% 14.14% 76.18% 95.54% 
Note: Number Encounters Sampled = Number of unique encounters ees a by the EQRO for medical record review validation; Number Medical Records 
Received = Number medical records submitted by MC+ MCO providers for validation; Number Validated = Number of encounters for which there was a matching 

diagnosis code, documentation or description in the medical record or on the claim form. Rate Validated of Medical Records Received = Number Validated/Number 

Medical Records Received; Actual Rate Validated = Number Validated/Number Encounters Sampled; LCL = Lower Confidence Limit at the 95% confidence level; UCL = 


Upper Confidence Limit at the 95% confidence level. 
Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, December 15, 2005. BHC, Inc. 2005 External 


Quality Review Validation of Encounter Data. 
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Figure 23 - Encounter Data Diagnosis Validation Rate, January |, 2005 — March 31, 2005 
* * 
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Note: * Indicates values are significant at the 95% level of significance, two-tailed z-test. See corresponding tables for 95% 
confidence intervals. 

Source: Missouri Department of Social Services, Division of Medical Services encounter claims extract file, 
December 15, 2005. BHC, Inc. 2005 External Quality Review Validation of Encounter Data. 


What Types of Errors Were Noted? 

An error analysis for procedure and diagnosis codes was conducted. For the diagnosis code in the 
medical record, the reasons for diagnosis codes not matching the SMA encounter claims extract file 
were incorrect information (n = 6). The diagnosis code listed did not match the descriptive 


information in the record. 


For the procedure code in the medical record, the reasons for procedure codes in the SMA 

encounter claims extract file not being supported by documentation in the medical record were 
missing information (n = 141) and incorrect codes (n = 103). Examples of incorrect information 
included: no code; incorrect codes; codes listed that were not supported; or codes that did not 


match the procedure description. 
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What Problems Are There With How Files Are Compiled and Submitted by 
the MCO? 


It was not possible to conduct the planned analysis of comparing MC+ MCO unpaid claims data to 
the SMA encounter claims extract file due to the variability across MC+ MCOs in the submission of 
data with correct Internal Control Numbers (ICNs) for the encounter data validation. Two plans 
submitted ICN data in the correct format and the EQRO compared the claims for these plans to all 


the claims in the sample from the SMA encounter claims extract file. 


What Are the Data Quality Issues Associated With the Processing of Encounter 
Data? 


There were several data quality issues with SMA and MC+ MCO encounter data identified during 
the course of conducting the EQRO. These issues are primarily related to the manner in which data 
are extracted from single and multiple databases. The MC+ MCOs did not submit unpaid encounter 
claims from the time period of interest. Various reasons were given for the MCOs’ inability to 
submit unpaid claims, including the claim that the MCOs do not retain this data after a denial has 
been issued and are therefore unable to provide unpaid encounter claims in the EQRO requested 
file layouts. This makes it untenable to identify errors of commission (unpaid claims in the SMA 


encounter claims file). 


4.5 Conclusions 


STRENGTHS 


|. All Dental, Home Health and Pharmacy claim type fields examined were |00.00% complete, 
accurate and valid for all MC+ MCOs. The SMA encounter claims data critical fields examined 
for accepted and paid claims of this type are valid for analysis. The next step would be to 
compare MC+ MCO paid claims to the SMA encounter claims database to identify the level of 
completeness of this data. 


2. For all MC+ MCOs, the Inpatient claim type critical fields examined were |00.00% complete, 
with the correct length of data 100.00% of the time. The Claim Type, Recipient ID, Admission 
Type, Bill Type, and Diagnosis Code fields were 100.00% valid for all MC+ MCOs. 


3. For all MC+MCOs, the Outpatient claim type critical fields examined were 100.0% complete 
with the correct length of data 100.0% of the time. The Claim Type, Recipient ID, Units of 
Service, Procedure Code, Place of Service fields were |00.00% valid. 


4. For all MC+ MCOs, the first Outpatient Diagnosis Code field was 100.0% complete, accurate 
and valid. 
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5. The examination of the level, volume, and consistency of services found significant variability 
between MC+ MCOs in the rate of each type of claim (Medical, Dental, Inpatient, Outpatient 
Hospital, Home Health, and Pharmacy), with no patterns of variation noted by MC+ Managed 
Care Region or type of MC+ MCO. 


6. Of the medical records received for review, the rates of matching on the diagnosis were 
99.01%. However, the medical records not submitted were unable to be validated. 


AREAS FOR IMPROVEMENT 


|. For the Medical claim type, there were invalid values for the Outpatient Units of Service and 
Outpatient Procedure Code fields, including fields of the wrong length. 


2. The Procedure Code field for all MC+ MCOs in all Claim Types included some invalid 
information. Most of this was due to blank fields. 


3. Across all MC+ MCOs, 52.0% of the medical records contained matching procedure codes. One 
MCO only match 42.0% of procedure codes in medical record review. 


4. For the Inpatient claim type critical fields, the Admission Date, Discharge Date, Patient Status, 
and Revenue Code fields contained invalid values. Invalid Admission Dates ranged from 
02/24/2004 — 12/31/2004. These findings may be a function of the manner in which the SMA 
encounter claims extract file was constructed. Invalid Discharge Date fields consisted of entries 
of “99999999” and invalid dates ranging from 04/01/2005 — 05/23/2005. 


5. For the Outpatient Hospital claim type, critical fields with invalid data were the Outpatient 
Procedure and Revenue Code fields. 


6. The match rates between the SMA database and MC+ MCO medical records for Medical claim 
type procedures were 52.0% respectively. Medical records that did not have procedure codes 
that matched the SMA encounter claims extract file were in error primarily due to missing or 
illegible information. 


7. lt was not possible to conduct the planned analysis of comparing MC+ MCO unpaid claims data 
to the SMA encounter claims extract file due to the variability across MC+ MCOs in the 
submission of data with correct Internal Control Numbers (ICNs) for the encounter data 
validation. Two plans submitted ICN data in the correct format and the EQRO compared the 
claims for these plans to all the claims in the sample from the SMA encounter claims extract file. 


RECOMMENDATIONS 


|. The SMA should prioritize examination of the level of completeness of the SMA Dental, Home 
Health, and Pharmacy paid encounter claim types relative the claims submitted by MC+ MCOs. 
These claim types are likely to be more complete and valid sources of data on which to base 
initial rate setting. This can be done once an identified method of obtaining paid and unpaid 
claims from MC+ MCOQOs is developed. 


2. It is recommended that the SMA institute additional edits for the Medical, Inpatient and 


Outpatient Hospital claim types to edit claims with blank fields or dummy values (e.g., “O00” and 
*99999999"’). 


3. MC+MCOs are required to submit Revenue Codes on the Outpatient Hospital (UB-92) claim 
file layout regardless of the Procedure Code. The SMA should institute edits for missing data. 
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4. 





It is recommended that future encounter data validation efforts explore the variability in the 
number of members who are enrolled with capitated providers to assess whether or not the 
variability in the proportion of each claim type or rate of encounters is related to the payment 
method. 


The SMA should continue to provide timely feedback to MC+ MCOs regarding the rate of 
acceptance of each claim type and the types of errors associated with rejected claims. 


Additional analysis on the rate of consistency of services should examine demographic (e.g., age 
and gender distribution), epidemiological (diagnostic variables), and service delivery (e.g., 
number of users per month, rate of procedures or claim types, units of service rates) 
characteristics to explain variation across MC+ MCOs or Regions. 


Medical record reviews should continue to be targeted toward validation of diagnosis and 
procedure codes. 


The MC+ MCOs should provide to the EQRO Internal Control Numbers (ICNs) in the form 
issued by the SMA when extracting encounter claims for medical record review purposes. 


The SMA should clarify the expectations for MC+ MCOs in the level of completeness, accuracy, 
and validity and which data fields are required (e.g., Diagnosis Code fields 2 through 5); provide 
timely feedback to MC+ MCOs when standards are not met; and develop corrective action 
plans when standards are not met within a reasonable amount of time established by the SMA. 


. MC+ MCOs will need to submit data to the EQRO in requested formats, using the field names 


and file formats requested. 
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5.0 MC+ MCO COMPLIANCE WITH MANAGED CARE 


REGULATIONS 





5.1 Purpose and Objectives 


The External Quality Review (EQR) is conducted annually in accordance with the 

Medicaid Program: External Quality Review of the Medicaid Managed Care Organizations Final Rule, 
42 CFR 438, Subpart E.” The original objective of this portion of the 2004 review was to analyze 
and evaluate the MC+ Managed Care Organizations (MC+ MCOs) to assess their level of 
compliance with federal regulations regarding quality, timeliness and access to health care services. 
In the two subsequent years, beginning in 2005, the objective is to complete a follow-up review to 
ensure improved and continued compliance with these regulations on the part of the MC+ MCOs. 
To complete this process, the “Monitoring Medicaid Managed Care Organizations (MCOs) and 
Prepaid Inpatient Health Plans (PIHPs): A Protocol for Determining Compliance with Medicaid 
Managed Care Regulations” (Compliance Protocol) requirements were applied to the review 
process, with an emphasis on areas where individual MCOs failed to comply or were in only partial 
compliance at the time of the prior review. Specifically, the MCOs were reviewed to assess MC+ 
MCO compliance with the federal Medicaid managed care regulations; with the State Quality 
Strategy; with the Missouri MC+ Medicaid Managed Care contract requirements; and the progress 


made in achieving quality, access, and timeliness to services from the previous review year. 
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5.2 Technical Methods 


PLANNING COMPLIANCE MONITORING ACTIVITIES 


Establishing Contact with the MC+ MCOs 

All MC+ MCOs were contacted during October 2005 to prepare them for the 2005 External 
Quality Review. All MC+ MCO quality management staff and/or plan administrators were contacted 
to discuss the onset of External Quality Review Organization (EQRO) activities and to schedule 
training teleconferences for mid-October. The MCOs were explicitly requested to have those staff 
or subcontractors available who were responsible for obtaining and submitting the data required to 
complete all validation processes. During the teleconferences, all aspects of the EQR were 


discussed and details provided regarding all data submissions that would be required. 


The training teleconference agenda, methods and objectives, and schedule were sent to all MC+ 
MCOs, with approval from the State Medicaid Agency (SMA), on October 18, 2005. SMA staff 
arranged to participate in these conference calls, which were held in late October, allowing time for 


presentation of information, clarification, and questions. 


Gathering Information on the MC+ MCO Characteristics 

During 2005 there were seven MC+ MCOs contracted with the State Medicaid Agency (SMA; 
Missouri Department of Social Services, Division of Medical Services; DMS) to provide MC+ 
Medicaid Managed Care in three Regions of Missouri. The Eastern MC+ Region included St. Louis 
City, St. Louis County, and eight surrounding counties. These MC+ Members were served by three 
MC+ MCOs: Community CarePlus (CCP), HealthCare USA (HCUSA), and Mercy Health Plan 
(MHP). The Western MC+ Region included Kansas City/Jackson County and eight surrounding 
counties. These MC+ Members were served by four MC+ MCOs: Children’s Mercy Family Health 
Partners (CMFHP), FirstGuard, Blue-Advantage Plus of Kansas City (BA+), and HealthCare USA 
(HCUSA). The Central MC+ Region included eighteen counties in the center of the state. These 
MC+ Member were served by two MC+ MCOs: Missouri Care (MOCare) and HealthCare USA 
(HCUSA). HealthCare USA operated in all three MC+ regions. 
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Determining the Length of Visit and Dates 
On-site reviews were conducted in one day, with several reviewers conducting interviews and 
activities concurrently. Interviews, presentations, and document reviews were scheduled 
throughout the day, utilizing different team members for Validating Performance Measures, 
Validating Performance Improvement Projects, Monitoring Medicaid Managed Care Organizations 
(MCOs) and Prepaid Inpatient Health Plans (PIHPs). The time frames for on-site reviews were 
determined by the EQRO and approved by the SMA before scheduling each MC+ MCO. The first 
week was spent reviewing the MC+ MCOs in Eastern MC+ Region. The second review week was 
spent in the Western MC+ Region. A final visit occurred with the MC+ MCO in the Central MC+ 
Region. The following schedule lists the dates of the on-site reviews: 

= February 28, 2006 — Community CarePlus 

» March I, 2006 — Mercy Health Plan 

» March 2, 2006 — HealthCare USA 

= March 7, 2006 — Children’s Mercy Family Health Partners 

» March 8, 2006 —Blue-Advantage Plus of Kansas City 

# March 9, 2006 — FirstGuard 

= March I5, 2006 — Missouri Care 


Reviewers 

Two reviewers conducted the Compliance Protocol activities, including interviews and document 
review. The External Quality Review Organization (EQRO) Project Director conducted backup 
activities and oversight to the Compliance Protocol team. The Assistant Project Director was 
conducting her second review. She had experience with the MC+ Managed Care Program 
implementation and operations, interviewing, program analysis, and Medicaid managed care 
programs in other states. The second reviewer participated in four previous MC+ Managed Care 
Program EQRs and on-site visits. This reviewer was knowledgeable about the MC+ Managed Care 
Program through her experience as a former SMA employee responsible for quality assessment and 
improvements, as an RN, and a consultant. Both reviewers were familiar with the federal 
regulations and the manner in which these were operationalized by the MC+ Managed Care 


Program prior to the implementation of the protocols. 
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Establishing an Agenda for the Visit 

An agenda was developed to maximize the use of available time, while ensuring that all relevant 
follow-up issues were addressed. A sample schedule was developed that specified times for all 
review activities including the entrance conference, document review, Validating Performance 
Improvement Project evaluation, Validating Performance Measures review, conducting the interview 
for the Compliance Protocol, and the exit conference. A coordinated effort with each MC+ MCO 
occurred to allow for the most effective use of time for the EQRO team and MC+ MCO staff. The 
schedule for the on-site reviews was approved by the SMA in advance and forwarded to each MC+ 
MCO to allow them the opportunity to prepare for the review. Appendix 8 provides a sample 


agenda for the on-site reviews. 


Providing Preparation Instructions and Guidance to the MC+ MCOs 

A letter (see Appendix 8) was sent to each MC+ MCO indicating the specific information and 
documents required on-site, and the individuals requested to attend the interview sessions. The 
MC+ MCOs scheduled their own staff to ensure that appropriate individuals were available and that 


all requested documentation was present during the on-site review day. 


OBTAINING BACKGROUND INFORMATION FROM THE STATE MEDICAID AGENCY 
Interviews and meetings occurred with individuals from the SMA from October 2005 through April 
2006 to prepare for the on-site review, and obtain information relevant to the review prior to the 
on-site visits. Individuals from the SMA included in these meetings were: 

# Sandra Levels, Director — Program Manager 

» Judith Muck — Assistant Deputy Director 

= Andrea Smith — Quality Program Liaison 

= Kimberly Carter, MC+ Managed Care QA & | Manager 
In November 2005, Compliance Review team members met with the SMA MC+ Managed Care QA 
& | Manager. The latest information on MC+ MCO compliance with the MC+ Medicaid Managed 
Care contract requirements was reviewed. All documentation gathered by the SMA was clarified 
and discussed to ensure that accurate interpretation of the SMA findings was reflected in the review 
comments and findings. The SMA staff continued to complete their review of MC+ MCO policy 
submissions. They provided periodic updates on approvals throughout the EQR preparation up to 
the beginning of the on-site review process. SMA expectations, requirements, and decisions specific 
to the MC+ Managed Care Program were identified during these meetings. 
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DOCUMENT REVIEW 

Documents chosen for review were those that best demonstrated the MC+ MCO’s ability to meet 
federal regulations. Certain documents, such as the Member Handbook, provided evidence of 
communication to members about a broad spectrum of information including enrollee rights and the 
grievance and appeal process. Provider handbooks and provider agreements were also reviewed to 
ensure that consistent information was shared regarding enrollee rights and responsibilities. SMA 
MC+ Medicaid Managed Care contract compliance worksheets, and specific policies that are 
reviewed annually or that are yet to be approved by the SMA, were reviewed to verify the presence 
or absence of evidence that required written policies and procedures existed meeting federal 
regulations. Other information, such as the Annual Quality Improvement Program Evaluation, was 
requested and reviewed to provide insight into the MC+ MCO’s report of their compliance with the 
requirements of the MC+ Medicaid Managed Care contract and the federal regulations. Grievance 
and Appeal policies, as well as a random selection of both member and provider records were 
reviewed at each on-site visit in an effort to obtain evidence of each MC+ MCO’s compliance with 
their own policy. When it was found that specific regulations were “Not Met” or “Partially Met,” 
additional documents were requested of each MC+ MCO. In addition, interview questions were 
developed to address the areas for which compliance was not fully established through the pre-site 


document review process. 


The following documents were reviewed for all MC+ MCOs: 


= State contract compliance ratings from 2005 and updated policies accepted through 
February 2006 

= Results, findings, and follow-up information from the 2004 External Quality Review 

» 2004 Annual MC+ MCO Evaluation, submitted April 2005 
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CONDUCTING INTERVIEWS 

After completing the initial document review, it was necessary to determine how policies were 
implemented, the progress that was made since the 2004 review, and what efforts were made to 
rectify areas where the MC+ MCOs were found “Not Met” or “Partially Met.” On-site interviews 
with MC+ MCO staff enabled reviewers to obtain a clearer picture of the degree of compliance 
achieved, as well as any corrective action taken by each MC+ MCO. This process revealed a wealth 
of information about the approach each MC+ MCO took to become compliant with federal 
regulations. It also provided evidence of systems where their members had quality and timely 
services, and the degree to which appropriate access was available. The interview provided 
reviewers with the opportunity to explore issues not addressed in the documentation, including 
follow-up from the 2005 EQRO evaluation. A site visit questionnaire was developed for each MC+ 
MCO based on their MC+ Medicaid Managed Care contractual compliance, issues identified for 


clarification, and information presented in their 2004 Annual Report. 


COLLECTING ACCESSORY INFORMATION 

Additional information used in completing the compliance determination included discussions with 
the EQR reviewers and MC+ MCO QI/UM staff regarding management information systems, 
Validating Encounter Data, Validating Performance Measures, and Validating Performance 
Improvement Projects. The review evaluated information from these sources to validate MC+ 
MCO compliance with the pertinent regulatory provisions within the Compliance Protocol. These 
findings were documented on the BHC MC+ MCO Compliance Review Scoring Form, and were 


used to make final rating recommendations. 


ANALYZING AND COMPILING FINDINGS 

The review process included gathering information and documentation from the SMA about policy 
submission and approval, which directly affected each MC+ MCO'’s contract compliance. This 
information was analyzed to determine how it related to compliance with the federal regulations. 
Next, interview questions specific to each MC+ MCO were prepared, based on the need to 
investigate if practice existed in areas where approved policy was not available, and if local policy 
and procedures were in use when approved policy was not complete. The interview responses and 
additional documentation obtained on-site were then analyzed to evaluate how they contributed to 
the MC+ MCO’s compliance. All information gathered was assessed, re-reviewed and translated 
into recommended compliance ratings for each regulatory provision. This information was 
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recorded on the MC+ MCO scoring form and can be found in the protocol specific sections of this 


section of the report. 


REPORTING TO THE STATE MEDICAID AGENCY 

In the April meeting with the SMA preliminary findings and comparisons to the ratings from the 
2004 review were presented. Discussion occurred with the SMA staff to ensure that the most 
accurate information was recorded and to confirm that a sound rationale was used in rating 
determination. The SMA approved the process and allowed the EQRO to finalize the ratings for 
each regulation. Sufficient detail is included in all worksheets to substantiate any rating lower than 


“Met.” Final worksheets were submitted to the SMA. The actual ratings are included in this report. 


COMPLIANCE RATINGS 

From December 2005 through February 2006, the MCO Compliance Review Scoring Form for each 
MC+ MCO was updated to reflect their current level of MC+ Medicaid Managed Care contract 
compliance. The Scoring Form continued to present a crosswalk of contract references that 
created compliance with each federal regulation. The SMA instructed the EQRO to utilize the 
Compliance Rating System developed during the previous review. This system was based on a 
three-point scale (“Met,” Partially Met,” “Mot Met”) for measuring compliance, as determined by the 
EQR analytic process. Appendix 9 contains the BHC MCO Compliance Review Scoring Form 
worksheet utilized for all MC+ MCOs. The determinations found in the Compliance Ratings 
considered SMA contract compliance, review findings, MC+ MCO policy, ancillary documentation, 
and MC+ MCO practices observed on-site. In some instances the SMA MC+ Medicaid Managed 
Care contract compliance tool rated a contract section as “Met” when policies were submitted, 
even if the policy had not been reviewed and “finally approved.” If the SMA considered the policy 
submission valid and rated it as “Met,” this rating was used unless practice or other information 
called this into question. If this conflict occurred, it was explained on the Compliance Review 


Scoring Form. The scale allowed for credit when a requirement was Partially Met. Ratings were 


defined as: 
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All documentation listed under a regulatory provision, or one of its 
components was present. MC+ MCO staff were able to provide responses to 
reviewers that were consistent with one another and the available 
documentation. Evidence was found and could be established that the MC+ 
MCO was in full compliance with regulatory provisions. 


Partially Met : There was evidence of compliance with all documentation requirements, but 
staff were unable to consistently articulate processes during interviews; or 
documentation was incomplete or inconsistent with practice. 


Not Met: Incomplete documentation was present and staff had little to no knowledge 
of processes or issues addressed by the regulatory provision. 





5.3 Findings 


ENROLLEE RIGHTS AND PROTECTIONS 

Subpart C of the regulatory provisions for Medicaid managed care (Enrollee Rights and Protections) 
sets forth 13 requirements of MCOs for the provision of information to enrollees in an 
understandable form and language: written policies regarding enrollee rights and assurance that staff 
and contractors take them into account when providing services; and requirements for payment and 
no liability of payment for enrollees. There were no items across MC+ MCOs that were rated as 
“Not Met” (see Table 47). Across all MC+ MCOs 75.82% of the regulations were “Met.” This is a 
significant improvement over the rate of 54.9% at the time of the 2004 EQR. Three MCOs 
(Children’s Mercy Family Health Partners, FirstGuard, Blue-Advantage Plus of Kansas City) were 
found to be 100% compliant. Two MCOs were less than 50% (Community CarePlus — 46.2%, Mercy 
Health Plan — 30.8%), however both showed improvement over their rating of 15.4% compliance 


rating at the time of the previous review. 


Huhne Group 
thea 7 ae 


6 Performance Management Solutions Group 165 
zee A division of Behavioral Health Concepts, Inc. 


Missouri MC+ Managed Care External Quality Review Section 5 
Report of Findings — 2005 MC+ MCO Compliance 


Table 47 - Subpart C: Enrollee Rights and Protections 
MC+ MCO All MC+ MCOs 


Vumber 
Partially | Number 
Federal Regulation eel =) HCUSA WW telerta- BA+ Met Not Met | Rate Met 
2 1 2 2 2 2 2 6 1 0 


438.100(a) Enrollee Rights: General Rule 85.7% 





438.10(b) Enrollee Rights: Information Requirements i dl 1 2 DD D 2 4 3 0 57.1% 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 2 2 2 2 2 7 0 0 100.0% 
438.10(c)(4,5) Language and Format: Interpreter 

Services il i if 2 2 Zz 2 4 3 0 57.1% 
438.10(d)(1)(i) Information Requirements: Format/Easily 

Understood 2 1 2 2 2 2 2 6 1 0 85.7% 


438.10(d)(1)(li)and (2) Information Requirements: Format 


Visually Impaired, and Limited Reading Proficiency it 2 2 2 2 Z 2 6 il 0 85.7% 
438.10(f) Information for All Enrollees: Free Choice, etc. 1 1 1 2 2 2 2 4 5 0 57.1% 
438.10 (g) Information to Enrollees: Specifics/Physician 

Incentive Plans 2 il 2 2 D2 2. 2 6 il 0 85.7% 
438.10(i) Special Rules: Liability for Payment/Cost 

Sharing 1 1 1 1 2 2 2 3 4 0 42.9% 
438.100(b)(2)(ili) Enrollee Rights: Provider-Enrollee 

Communications 2 2 2 2 22 2 2 7 0 0 100.0% 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance 

Directives 1 1 2 2 2 2 2 S 2 0 71.4% 
438.100(b)(3) Right to Services i il 2 1 2 2 Z 4 3 0 57.1% 
438.100(d) Compliance with Other Federal/State Laws 2 2 2 2 2 2 2 7 0 0 100.0% 
Number Met 6 4 9 11 13 13 a3 69 22 0 75.82% 
Number Partially Met 7 9 A 2 ) 0 ) 

Number Not Met 0 0 0 0 0 0 0 

Rate Met 46.2% 30.8% 69.2% 84.6% 100.0% 100.0% 100.0% 





Note: 0 = Not Met; |= Partially Met ; 2 = Met 


Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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All MC+ MCOs had procedures and practices in place to ensure that members were addressed in 
their prevalent language [438.10(c)(3)]; that members are treated with respect and dignity and 
receive information on available treatment options and alternatives [438.100(b)(2)(iii)]; and the MC+ 


MCO is in compliance with other state requirements [438.100(d)]. 


A number of MC+ MCOs (Children’s Mercy Family Health Partners, Missouri Care, Community 
CarePlus, Blue-Advantage Plus of Kansas City) utilized EQR tools, including the MCO Compliance 
Review Scoring Form, to assist them in ensuring completion of required policy as well as meeting 
the requirements of the federal regulations. Improvement was noted in the attention the majority 
of the MC+ MCOs gave to meeting all standards of compliance. Tracking systems were put in place, 
and in some situations staff members were assigned to monitor compliance issues. The MC+ MCOs 
stressed their heightened awareness of the need for positive interdepartmental communication. 
These efforts focused on strengthening communication to enhance the organizations’ ability to serve 


members needs. 


One MC+ MCO (Children’s Mercy Family Health Partners) initiated a Member Advisory Committee 
to provide insight into the issues faced by members in trying to obtain healthcare services. The 
MC+ MCO incorporated member suggestions into their operations and marketing materials. These 
activities were indicators of the MC+ MCO’s commitment to member services and to ensuring 


members had quality healthcare. 


BEHAVIORAL HEALTH 

All MC+ MCOs continued to operate programs for the provision of behavioral health services. Six 
of the MC+ MCOs subcontract with Behavioral Health Organizations (BHOs) for these services. 
One MC+ MCO (Missouri Care) has moved the supervision and delivery of this service in-house. 
One MC+ MCO (FirstGuard) underwent a transition in ownership during the 2005 review period, 


which also led to the provision of behavioral health services by a new subcontractor. 


All MC+ MCOs provided active oversight, if not direct involvement, of their behavioral health 
subcontractors. Behavioral Health Services have evolved into an important resource for MC+ 
Medicaid Managed Care members. One MC+ MCO (Mercy Health Plan) integrated their care 
management system with that of their Behavioral Health Organization (BHO). This ensured that all 


cases with co-morbidity were targeted, and that both the PCP and behavioral health provider were 
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aware of services and medications members receive. This action allowed for a more holistic 
approach to the delivery of healthcare services for members. A number of the MC+ MCO BHOs 
(Community CarePlus, Mercy Health Plan, HealthCare USA, New Directions Behavioral Health, 
CommCare and Cenpatico) approved the use of in-home services to reach members who would 
not attend appointment in an office setting. This not only ensured that members obtained the help 
they needed, but also prevented missed appointment for providers. One MC+ MCO (New 
Directions Behavior Health) continued to contract with a provider agency that delivered short-term 
intensive in-home services in an effort to avert crisis that may lead to inpatient treatment, and to 
work with members to utilize all available community resources. Two MCOs (Mercy Health Plan, 
HealthCare USA) reported on initiatives to engage members who were pregnant, in an attempt to 
identify any mental health issues that might affect the mother or baby. These efforts also focused on 
prevention of postpartum depression. One MC+ MCO (Children’s Mercy Family Health Partners) 
described an initiative where in-home services were provided to members following any inpatient 
treatment to ensure effective follow-up services. The BHO contracted with specific providers who 
were skilled at working in intensive in-home settings. The BHO absorbed the cost of unreimbursed 
services, such as after-hours telephone support, in an effort to reduce readmissions for these 
members. MC+ MCOs and BHOs described a number of interventions that met members’ needs, 
but were extraordinary in normal Medicaid programs. This reflected a level of performance 


indicative of their strong commitment to access and quality services for all members. 


QUALITY ASSESSMENT AND PERFORMANCE IMPROVEMENT: ACCESS STANDARDS 
Subpart D of the regulatory provision for Medicaid managed care sets forth |7 regulations governing 
access to services. These regulations call for: the maintenance of a network of appropriate 
providers including specialists; the ability to access out-of-network services in certain circumstances; 
adequate care coordination for enrollees with special healthcare needs; development of a method 
for authorization of services, within prescribed timeframes; and the ability to access emergency and 
post-stabilization services. There were no items rated as “Not Met” (see Table 48). Across all 
MC+ MCOs, 78.99% of the regulations were “Met,” which is a substantial improvement over the 
rate of 60.5% at the time of the 2004 EQR. Three of the MCOs (Children’s Mercy Family Health 


Partners, Blue-Advantage Plus of Kansas City, FirstGuard) were found to be 100% compliant. 
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All MC+ MCOs had policies and practice that reflected the members’ right to a second opinion and 
a third opinion if the first two disagreed [438.206(b)(3)]. Another area where all MC+ MCOs were 
100% compliant was in provider cultural competency. Evidence of this included an MC+ MCO 
(Mercy Health Plan) who recruited a home health provider who spoke Vietnamese solely for a 
member. Throughout this review period, all health plans reported incidents where they found 
providers who were familiar with members’ cultural and language needs. Sensitivity to and respect 
for members’ cultural needs was an area where the MC+ MCOs excelled. All MC+ MCOs were 
fully compliant in having SMA approved notifications of adverse actions [438.210(c)]. There were no 
identified incidents of incentivizing staff or contractors for utilization management decisions that 
were in the favor of the MC+ MCOs. All policies and practices in this area [438.210(e)] were 


compliant. 


Some problems in the area of access to care remained evident in this review. Required 
documentation and approved policies did not exist in all areas. One Member Handbook (Mercy 
Health Plan) continued to contained confusing language concerning the need for a primary care 
physician (PCP) referral to obtain OB/GYN services. The MC+ MCO corrected this language, but 
not during the 2005 review year. Four MC+ MCOs (Community CarePlus, Mercy Health Plan, 
HealthCare USA, , Missouri Care) had outstanding policy or Provider Manual language in the area of 
emergency and post-stabilization services [438.114]. One MC+ MCO (First Guard) underwent a 
transition in ownership, which created number of changes, particularly in their claims and payment 
system. The MC+ MCO made extraordinary efforts to ensure that the problems they experienced 
remained transparent to members. Although they did struggle with provider issues during this 
period, they were able to resolve these problems. The MC+ MCO had strong relationships with 


their providers and maintained communication with them throughout the transition. 


The MC+ MCOs made a concerted effort to ensure that members had appropriate and timely 
access to services. The MC+ MCOs also experienced issues with closed panels, many physicians 
were not able to take new patients. Additionally, they continued to express concern over the 
shortage of specialists in the areas of orthopedic surgery, pediatric neurology, and child/adolescent 
psychiatrists. All MC+ MCOs reported utilizing out-of-network providers and often paying 
commercial or higher rates to obtain these services. One MC+ MCO (Children’s Mercy Family 
Health Partners) had a number of specialists who requested that they not be included on the MC+ 


MCO’s published network, but readily agreed to service MC+ Managed Care members. A number 
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of the MC+ MCOs (Mercy Health Plan, HealthCare USA, Missouri Care) continued to partner with 
the teaching hospitals in the Regions, in order to increase their available surgical and specialist 


capacity. 


Two MC+ MCOs (FirstGuard, Missouri Care) underwent transitions in their behavioral health 
service system during the 2005 review period. One MC+ MCO (Missouri Care) no longer uses a 
subcontracted network for behavioral health. This MC+ MCO recognized a number of advantages 
in directly supervising the provision of behavioral health services. They contracted with the majority 
of the active providers previously utilized by the subcontractor. They maintained the same toll-free 
telephone number for member access, and completed provider training prior to the change. Some 
of the benefits identified included: reducing the use of inpatient treatment; more timely and 
complete prior authorizations; and improved case management, particularly for members who 


require both physical and mental health treatment. 


At the time the second MC+ MCO (FirstGuard) changed ownership, they also transitioned to a new 
behavioral health provider, who was a subsidiary of the new owner. During a 60-day transition 
period, the MC+ MCO required no prior authorizations for services to eliminate this possible 
barrier to member services. They maintained most of the providers who contracted with the 
previous subcontractor, as well as the same toll-free member access telephone number. The MC+ 
MCO received no member grievances associated with transition. Members were notified of any 


required provider changes. However, there appeared to be no adverse therapeutic effects of this 


transition. 
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Table 48 - Subpart D: Quality Assessment and Performance Improvement: Access Standards 


MC+ MCO All MC+ MCOs 
| ee : | 7 a a a Number | , 
Number Pescasidd | Number 


Met | Not Met Rate Met 


A38206(0\iiv) Avalablityerservees: PoverNewok |?) dR) gL | 0) ST 
438.206(6)(3) Second Opinions ek gg 00.0% 


ar-Yel-Ve-] | Reg uleuOn HCUSA WW oler-Ta-) FHP ze BA+ Met 





438.206(b)(5) Out of Network Services: Cost Sharing | aaa | | ak gr 
438.208(c)(2) Provider Services: Cultural Competency | 2 | | | | 100.0% 
438.208(c)(1) Care Coordination: Identification =| a | | ga | 7H 
438.208(c)(3) Care Coordination: Treatment Pans | a ata | | a | gk ray 
438.210(b) Authorization ofServices | a | kg a | 1% 
438.210(a) Timeframes for Decisions, Expedited Authorizations | 22a) | I 








SE EE ER 
a Met 0 78.99% 
a 
NumberNotMetSSSC*~—“~*~*~“‘*S*S*S*S*S*S*S*S*~<“s*~SS*SSCSCSC“‘“OCSSCOTTSCOTSSCC 


Note: 0 = Not Met; I= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A 
protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs 
Protocols. 
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QUALITY ASSESSMENT AND PERFORMANCE IMPROVEMENT: STRUCTURE AND 
OPERATION STANDARDS 


There are 10 Structure and Operations Standards for ensuring compliance with State policies and 
procedures for the selection and retention of providers, disenrollment of members, and 
accountability for activities delegated to subcontractors. There were no items across MC+ MCOs 
that were rated as “Not Met.” Across MC+ MCOs, 88.6% of the regulations were “Met,” which is 


an improvement over the rating of 65.7% from the 2004 EQR (see Table 49). 


The Provider Services departments of all MC+ MCOs exhibited a sound and thorough 
understanding of the requirements for provider selection, credentialing, nondiscrimination, 
exclusion, and MC+ Medicaid Managed Care requirements. Three of the MC+ MCOs (Children’s 
Mercy Family Health Partners, Blue-Advantage Plus of Kansas City, FirstGuard) were 100% 
compliant with all regulations. Two additional MC+ MCOs met 90% of the regulations. Four of the 
individual regulations were 100% met. These included Provider Selection [438.214(d) and 
438.214(e)]. The staff at each MC+ MCO understood the requirements for disenrollment. They 
were 100% “Met” for the applicable regulations for timeframes [438.56(e)]. Six of seven (87.5%) 
met all regulations for disenrollment procedures. All MC+ MCOs (100%) had appropriate grievance 
systems in place that met the requirements of this regulation [438.228]. A number of the MC+ 
MCOs (Mercy Health Plan, Blue-Advantage Plus of Kansas City, and FirstGuard) described 
credentialing and recredentialing policies that exceeded the requirements of the regulations. 
Providers were willing to submit to these stricter standards to maintain network qualifications in 
both the MC+ MCO and commercial networks of these MC+ MCOs. Overall, six of seven (85.7%) 


of the MC+ MCOs had all required policies and practices in place regarding credentialing. 


All MC+ MCOs understood the required oversight of subcontractors. The compliance rate for this 
regulation [438.230(a,b)] improved from the 2004 rate of 14.3%, to the 2005 rate of 71.4%. The 
MC+ MCOs that were “Partially Met” (Mercy Health Plan, HealthCare USA) had policy or 


subcontractor agreements that were not yet approved by the SMA. 


All deficiencies for Structure and Operation Standards related to a lack of submitted or approved 
policies or subcontractor agreements. The MC+ MCOs exhibited a significantly improved 


understanding and attention to these details and requirements during this review. 
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Table 49 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation Standards 


| | All MC+ MCOs 


PP pp E  NGmber .  — 
aye ae Seabee Number 
ro (-Va-JMst-te[ViF-Latey a HCUSA | MOCare et N 


fo} am (= 5 Rate Met 


438.214(d) Provider Selection: Excluded Providers | || 100.0% 


438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 
limitations 1 2 i 2 2 2 ? 5 2 71.4% 


438.56(d) Disenroliment: Procedures ES ES 
438.228 Grievance System 2 | 2 | a | kt | gg | 100.0% 


8 0) 88.6% 
Number Partially Met 


pee ee ee 
e+ 4 2 2 tt 8 


Note: 0 = Not Met; I= Partially Met ; 2 = Met 





Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for determining compliance 
with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 External Quality Review Monitoring MCOs Protocols. 
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QUALITY ASSESSMENT AND PERFORMANCE IMPROVEMENT: MEASUREMENT AND 
IMPROVEMENT 


There are 12 Measurement and Improvement Standards addressing the selection, dissemination, and 
adherence to practice guidelines; the implementation of performance improvement projects; the 
calculation of performance measures; the evaluation of the availability of services and assessment 
techniques for enrollees with special healthcare needs; and the maintenance of information systems 
that can be effectively used to examine service utilization, grievances and appeals, and disenrollment. 
All items were either “Met” or “Partially Met” for compliance with Measurement and Improvement 
(see Table 50). A total of 83.1% of the criteria were “Met” by the MC+ MCQOs, which continues to 
indicate improvement in meeting federal requirements, over the 2004 rate of 61.0%. Four MC+ 
MCOs (Mercy Health Plan, Children’s Mercy Family Health Partners, FirstGuard, Blue-Advantage 


Plus of Kansas City) met all the requirements (100%) in this area. 


The area of practice guidelines was problematic for two MC+ MCOs (Community CarePlus and 
HealthCare USA). Both had new Medical Directors, who identified resistance on the part of the 
medical community in the St. Louis area to the acceptance or implementation of practice guidelines. 
The specific requirements of the regulations were related to both MC+ MCOs during the on-site 
review. The majority of the MC+ MCOs (71.4%-85.7%) met all the requirements for adopting, 
disseminating and applying practice guidelines. In the Western MC+ Medicaid Managed Care region, 
staff from the MC+ MCOs met with a quality enhancement group in the healthcare community 
(Kansas City Quality Improvement Consortium). Regional standards and practices were discussed 
and regionally specific standards, that met or exceeded nationally accepted guidelines, were 
developed. All MC+ MCOs related that they expected providers to use the practice guidelines 
combined with their experience and patient knowledge in their decision-making. When conflicts 
occurred, the Medical Director reviewed the situation and consulted with the provider in an effort 


to ensure that the services that were provided were in the members’ best interest. 


All MC+ MCOs (100%) used nationally accredited criteria for utilization management decisions 
[438.240(b)(3)]. The tools the MC+ MCOs reported using included the InterQual Clinical 
Decision Support Tool and the Milliman Care Guidelines. Both sources provided evidence- 
based criteria and best practice guidelines for healthcare decision-making. One MC+ MCO 
(Missouri Care) had adopted the use of LOCUS/CALOCUS (Level of Care Utilization System/Child 
and Adolescent Level of Care Utilization System) for utilization management decisions in the 
provision of behavioral health services. The MC+ MCOs staff was able to articulate how they 
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utilized these tools and apply them to member healthcare management issues. They stated that they 
used all the information available to them ensure evidence-based practice that ensured member 


safety while controlling medically unnecessary care. 


The MC+ MCOs were actively involved in developing and improving their Quality assessment and 
Improvement Programs. Two of the MC+ MCOs (FirstGuard, Children’s Mercy Family Health 
Partners) utilized community based advisory boards, one of which (Children’s Mercy Family Health 
Partners) included members. These groups assisted the MC+ MCOQOs in assessing member needs 
and barriers to services. Both MC+ MCOs utilized the recommendations of these groups in their 
operations, member information, and daily activities. All MC+ MCOs developed internal systems 
for monitor, analysis and evaluation of their own programs. Six of seven (85.7%) had a program and 
all required policy and procedures in place to meet the requirements of the federal regulations 


[438.240(a)(1)]. 


The Compliance Protocol seeks to ensure that MC+ MCOs comply with the requirements of the 
sections of the protocol involving Validating Performance Improvement Projects, Validating 
Performance Measures, Validating Encounter Data, and Health Information Systems. Detailed 
findings and conclusions for these items are provided in previous sections of this report and within 


the MC+ MCO summaries. 
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Table 50 - Subpart D: Quality Assessment and Performance Improvement: Measurement and Improvement 


: MC+ MCO | All MC+ MCOs 
Number 


: Number | Partially | Number 
Federal Regulator ccp | MHP HCUSA | MOCare dalig BA+ Met Met ARTS || EMS Leet 


A a 
SAP) Paciee Meer APIA tt et tt Pt Tae 


438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 
Improvement and PIPs 1 2 1 1 2 2 2 3 57.1% 


438,240(5)3) QAP: Basic lements/Over and Under uizaton gg ggg scam 
a 


Number Met tt tt 13 83.1% 
ABS Te) cee A OS (a 
Ls (ee ee 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least annually, the impact and effectiveness of each MCO's quality assessment and performance improvement 
program. The regulation refers to the state QA & | program review process and is not applicable to External Quality Review of the MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot 
the MC+ Managed Care Program. 


0 = Not Met; I= Partially Met ; 2 = Met 





Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for determining compliance 
with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version |.0.; BHC, Inc., 2004 External Quality Review Monitoring MCOs Protocols. 
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GRIEVANCE SYSTEMS 

Subpart F of the regulatory provisions for Medicaid managed care (Grievances and Appeals) sets 
forth 18 requirements for notice of action in specific language and format requirements for 
communication with members, providers and subcontractors regarding grievance and appeal 
procedures and timelines available to enrollees and providers. All MC+ MCOs excelled (96.0%) in 
their compliance with the regulations related to grievances and appeals (see Table 51). There were 
no items rated as “Not Met.” Four MC+ MCOs (HealthCare USA, Children’s Mercy Family Health 
Partners, Missouri Care, and Blue-Advantage Plus of Kansas City) were found 100% in completing 


required policy, procedure, and practice in their Grievance Systems. 


Grievance and Appeal reports for both members and providers were reviewed from the first 
quarter of 2005, as submitted to the SMA. The MC+ MCOs reported radically different numbers 
and types of concerns. The number of member grievances and appeals varied from one MC+ MCO 
(Community CarePlus) reporting 5, to another MC+ MCO (HealthCare USA) reporting 270 in the 
Eastern MC+ Medicaid Managed Care region. Provider complaints, grievances, and appeals varied 
from one MC+ MCO (Mercy Health Plan) reporting a total of |7, to another MC+ MCO 
(Healthcare USA) reporting |835 for the Eastern Region. A total of 495 member grievance and 
appeals were reported, and a total of 3112 provider complaints, grievances, and appeals were 


reported from all three MC+ Medicaid Managed Care regions for this three month period. 


In analyzing the Grievance System report, the most frequent issues included: 


Member - Grievances and Appeals Provider — Complaints, Grievances, and 
Appeals 


Transportation Authorizations — Denied/Late/None 
Prescription Drug Issues Billing Problems 

Appointment Contractual Issues 
Availability/Continuity of 

Treatment 


Treatment by Provider/Staff e Untimely Submission of Claims 
Service Category/Prior Auth. e Uncovered Benefit 

(denial) 

Claims Issue/Uncovered Benefit e Additional Information Required 
Inability to Find PCP/Specialist — or e Medical Necessity Question 
Obtain an Appointment 

State Fair Hearing Request 
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The largest number of member grievance/appeals revolved around transportation issues. The 
largest number of provider complaints/grievances/appeals included authorization issues and untimely 
submission of claims. The majority of the claims were the result of payment disputes, although a 
number of grievances and appeals filed by providers did dispute decisions that appeared to affect the 


quality of care received by members. 


A random selection of both member grievance and appeals, and provider complaints, grievances, and 
appeals were identified for on-site review. The random list was presented to each MC+ MCO at 
the beginning of the on-site review. The results of these reviews were presented during the Exit 
Interview. The review did identify problems in several of the records reviewed. Two MC+ MCOs 
(Mercy Health Plan, FirstGuard) exhibited problems, such as inadequate correspondence or 
incomplete information to substantiate the determinations made. One MC+ MCO (Community 
CarePlus) had inadequate documentation in the records reviewed. This MC+ MCO recognized the 
problems that existed in their system and made significant changes throughout the 2005 review 
period. They did share additional cases as examples of how records were kept, and internal 


reporting requirements that were in place at the time of the on-site review. 


There were very few deficiencies in the Grievance System policy submission. The only regulations 
that were rated as “Partially Met” were the result of inadequate record keeping, and 
correspondence that was not available to meet the MC+ MCOs’ own policy requirements. 
Appropriate practice for addressing member grievance and appeals, and provider complaints, 


grievances and appeals appeared to be in place for all MC+ MCOs. 
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Table 51 - Subpart F: Grievance Systems 


MC+ MCO All MC+ MCOs 





Number Number Not 
Federal Regulation ccP HCUSA WM leleryas) BA+ Partially Met Met Rate Met 


456.4UZ(a) Grlevance and Appeals: General 





Requirements 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4UZ(D)(L1) Grlevance system: FUINg REQUIFeEMeNtS - 

Authority 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4UZ(D)(Z) Grievance system: FHINg REQuiIrements - 

Timing 2 p 2 2 2 2 2 7 0 0 100.0% 
456.4UZ(D)(5) Grievance system: FHINg REQUIFeEMeENtS - 

Procedures 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4U4(da) UlleVdalCe SYSLeIM: INOLICE OF ACLION - 

Language and Format 2 2 2 2 2 2 2 7 0 0 100.0% 
438.404(b) Notice of Action: Content 2 2 2 2 2 2 2 7 0 0 100.0% 
438.404(c) Notice of Action: Timing 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4U0(da) HAandailng OF Grievances and APPedls: 

General Requirements 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4U0(D) Manalng Ol Grievance and APPed!lS: BpPpeCidl 

Requirements for Appeals 2 2 2 2 2 2 2 7 0 0 100.0% 
438.408(a) Resolution and Notification: Basic Rule 2 1 2 2 ? 1 2 5 > 0 71.4% 
456.4U6(D,C) ReSOIUTION and NOUTIHICATION: Grievances 

and Appeals - Timeframes and Extensions 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4U6(Q)(€) ReESOIUTION anQ NOTITICATION: Grievance 

and Appeals - Format and Content of Notice 2 2 2 2 2 2 2 7 0 0 100.0% 
456.4U6(I) ReESOIULION aNd INOLITICALION: Grlie@VaNnCces anda 

Appeals - Requirements for State Fair Hearings 2 2 2 2 2 D 2 7 0 0 100.0% 
438.410 Expedited Resolution of Appeals 2 2 2 2 2 2 2 7 0 0 100.0% 
456.414 InrOrmation about tne Grievance system to 

Providers and Subcontractors 2 1 2 2 2 1 2 5 2 0 71.4% 
438.416 Recordkeeping and Reporting Requirements if 2 2 2 2 2 2 6 1 0 85.7% 
456.4ZU CONLUNUALION OF BENeMlLS WNIE APPedl/ Fall 

Hearing Pend 2 2 2 2 2 2 Z 7 0 0 100.0% 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 > 2 2 2 > 7 0 0 100.0% 


Note: 0 = Not Met; |= Partially Met, 2 = Met 


Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans 
(PIHPs): A protocol for determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final Protocol Version !|.0.; BHC, Inc., 2005 External Quality Review 
Monitoring MCOs Protocols. 
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5.4 Conclusions 


Across all MC+ MCOs there was a substantial improvement in the area of compliance with federal 
regulations. There were no regulations rated as “Not Met.” All individual regulations were rated as 
“Met” or “Partially Met.” Two MC+ MCOs were 100% compliant with all requirements. Three 
were 100% compliant with the regulations related to Enrollee Rights and Protections. Three were 
100% compliant with Access Standards. In the area of Structure and Operations all MC+ MCOs 
were at 60% compliance or higher. Three werel00 % and two additional MC+ MCOs were 90% 
compliant. Four of the MC+ MCOs met all required regulations in the area of Measurement and 
Improvement. The area of Grievance systems was also substantially compliant across all the MC+ 
MCOs. Four were 100% compliant, one was 94.4%, and two were 88.9%. A\ll sources of available 
documentation, interviews, and observations at the on-site review were used to develop the ratings 
for compliance. The EQRO comments were developed based on review of this documentation and 
interview responses. Several of the MC+ MCOs made it clear that they used the results of the 
prior EQR to complete and guide required changes. One MC+ MCO (Community CarePlus) 
significantly improved and stated that they utilized the compliance protocol as a tool to develop 
their performance and improve services to members. This MC+ MCO achieved improved 
compliance in every category. The following summarizes the strengths, areas for improvement, and 
recommendations based on the findings utilizing the Protocol for Determining Compliance with 


Medicaid Managed Care Regulations. 


STRENGTHS 


|. Two of the MC+ MCO “met” (100%) applicable federal regulations and State compliance 
requirements. There were no regulations rated as “Unmet” for any of the MC+ MCOs. The 
ratings for every section of the protocol indicated an improvement in all areas of policy and 
procedure submission and approval, as well as practice. 


2. Three of the I3 regulations for Enrollee Rights and Protections were 100% “Met.” 
Communicating MC+ Members’ rights to respect, privacy, and treatment options, as well as 
communicating, orally and in writing, in their own language or with the provision of interpretive 
services is an area of strength for all MC+MCOs. The MC+ MCOs maintained an awareness of 
and appropriate responses to cultural and language barriers concerning communication in 
obtaining healthcare. The MC+ MCOs responded to physical, emotional and cultural barriers 
experienced by members with diligence and creativity. 


3. The MC+ MCOs were aware of their need to provide quality services to members in a timely 
and effective manner. Vhere there were issues with access to services, the MC+ MCOs 
responded quickly and effectively. 
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4. Four of the |7 regulations for Access Standards were 100% “Met.” These included the 
regulations regarding second opinions, provider cultural competency, notices for adverse 
actions, and compensation of utilization management activities. MC+ MCOs monitored high risk 
MC+ Members and had active case management programs in place. Each MC+ MCO described 
measures to identify and work with MC+ Members who had special healthcare needs exceeding 
the requirements of the MC+ Medicaid Managed Care contract. 


5. Four of the 10 regulations for Structure and Operations Standards were 100% “Met.” These 
included provider selection, both excluded providers and state requirements; grievance systems; 
and subcontractual relationships and delegation. The MC+ MCOs had active mechanisms for 
oversight of all subcontractors in place. All MC+ MCOs improved significantly in compliance 
with this set of regulations. 


6. Three of the |2 regulations for Measurement and Improvement were 100% “Met.” Three of 
the seven MC+ MCOs met all of the regulatory requirements. Five of the seven MC+ MCOs 
adopted, disseminated and applied practice guidelines to ensure sound healthcare services for 
members. The MC+ MCOs used their health information systems to examine the appropriate 
utilization of care using national standard guidelines for utilizations management. The MC+ 
MCOs were beginning to utilize the data and demographics in their systems to track and trend 
information on members to assist in determinations of risk and prevention initiatives. Several 
MC+ MCOs began using member and community based quality improvement groups to assist in 
determining barriers to services and methods to improve service delivery. The Provider Service 
or Relations departments of the MC+ MCOs exhibited a commitment to relationship building, 
as well as monitoring providers to ensure that all standards of care were met and that good 
service, decision-making, and sound healthcare practices occurred on behalf of MC+ Members. 


7. Fourteen of the |18 regulations for Grievance Systems were 100% “Met.” Four of the MC+ 
MCOs were 100% compliant with the requirements for policy, procedure and practice in the 
area of Grievance Systems. 


8. MC+ MCOs remained invested in developing programs and providing services beyond the strict 
obligations of the contracts. Preventive health and screening initiatives exhibited a commitment 
to providing the best healthcare in the least invasive manner to their MC+ Members. 
Partnerships with local universities and medical schools provided opportunities to obtain 
cutting-edge and occasionally experimental treatment options, which would not otherwise be 
available to MC+ Members. 
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AREAS FOR IMPROVEMENT 


All MC+ MCOs did not have their written policies and procedures completed to ensure that a 
consistent application of contractual requirements and federal regulations was occurring. 


MC+ MCOs continued to struggle with recruitment of certain specialty physicians. Throughout 
discussions with MC+ MCOs the lack of orthopedic surgeons, neurosurgeons and 
child/adolescent psychiatrists was identified as a problem. The MC+ MCOs have made 
accommodations to ensure that members received the services required. Through the use of 
advance practice nurses, silent physician partners, cooperative agreements with medical schools, 
and the willingness to reimburse at commercial insurance rates, the MC+ MCOs attempt to 
ensure that members have access to these services. MC+ MCOs expressed continued concern 
for improvement in this area. 


MC+ MCOs identified the need for additional dental providers. Recruitment was largely 
delegated to subcontractors. Becoming actively involved in recruitment activities would benefit 
members and improve the quality of and access to care. 


The use of data for quality improvement purposes and examination of healthcare outcomes was 
just beginning. Continued growth in the utilization of all of the data available to drive healthcare 
practice and initiatives is required to improve quality and access to care. 


RECOMMENDATIONS 


Continue to distribute the completed compliance tools to MC+ MCOs to ensure recognition of 
the policies and procedures that must be completed and approved to achieve compliance with 
federal regulations. 


MC+ MCOs must continue to recognize the need for timely submission of all required policy 
and procedures. The majority of the MC+ MCOs put a tracking or monitoring system into 
place to ensure timely submission of documentation requiring annual approval. These systems 
must be maintained to ensure that this process remains a priority for all MC+ MCOs. 


MC+ MCOs identified the need for continuing to monitor provider availability in their own 
networks. Although most MC+ MCOs had the number of primary care physicians (PCPs) and 
specialists required to operate, they admitted that many of these PCPs had closed panels and 
would not accept new patients. Ensuring that there is adequate access for all members, 
including new members, should be a priority for all MC+ MCOs. 


MC+ MCOs identified improvement in their Quality Assessment and Improvement programs, 
and how this enhanced their ability to provide adequate and effective services to members. 
These efforts must be relentlessly continued to ensure that the organizations remain aware of 
areas for growth and improvement. These efforts ensure that the quality, timeliness and access 
to care required for member services is maintained at an exceptional level. 


Chee Performance Management Solutions Group 182 


lone Group 
bor oo ba 


oe ran BE A division of Behavioral Health Concepts, Inc. 


Missouri MC+ Managed Care External Quality Review Section 5 
Report of Findings — 2005 MC+ MCO Compliance 


(This page intentionally left blank.) 






e.. cituieaneee Performance Management Solutions Group 183 
ene A division of Behavioral Health Concepts, Inc. 


Missouri MC+ Managed Care External Quality Review Section 6 
Report of Findings — 2005 Community CarePlus 


~) 5 OS BLOCK! 


OOD CY CORD iS A Oy.N 45) td 





Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 184 





Missouri MC+ Managed Care External Quality Review Section 6 
Report of Findings — 2005 Community CarePlus 


(This page intentionally left blank.) 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 185 





Missouri MC+ Managed Care External Quality Review Section 6 
Report of Findings — 2005 Community CarePlus 


6.0 - Ww, 
( A.R E P L.U S 


6.1 Performance Improvement Projects 





The previous sections of the 2005 EQRO report present the purpose and objectives, technical 
methods, procedures for evaluation, and MCO to MCO comparisons of analyses, findings, and 
recommendations for each of the protocols based on the aggregate analysis. This section of the 
EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed 
Care Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


METHODS 
Document Review 
Community Care Plus supplied the following documentation for review: 
= Performance Improvement Project 2005: Early Intervention in Prenatal Care 


Management and the Relationship to Very Low Birth Weight Babies 
= Performance Improvement Project 2005: Emergency Room Utilization 


Interviews 

Interviews were conducted with the project leaders for each Performance Improvement Project 
(PIP) by the EQRO team on February 28, 2006 during the on-site review, and included the 
following: 


Marcia Albridge — Director, Business Development 
Sue Danis — Manager, Performance Improvement 
Beverly Thompson — Director, Medical Management 
Judy Milam — Manager, Claims/Customer Service 
Robin Woolfolk — Manager, Customer Service 


C maine moan Performance Management Solutions Group 


tiene Groupe 


ross bee en m BE A division of Behavioral Health Concepts, Inc. 186 


Missouri MC+ Managed Care External Quality Review Section 6 
Report of Findings — 2005 Community CarePlus 


The interviewees shared information on the validation methods, study design, and findings. 
Technical assistance regarding study design and presentation of findings was provided by the 
EQRO. The following questions were addressed: 


=» Who was the Project Leader? 

= How was the topic identified? 

= How was the study question determined? 

=» What were the findings? 

=» What was the intervention? 

= What was the time period of the study? 

=» Was the intervention effective? 

» What does Community CarePlus want to study or learn from their PIPs? 
The maturity of the PIPs submitted for validation was not complete at the time of the initial 
request for this information in October 2005. The MCO was instructed that they could submit 
additional information that included outcomes of the intervention. The final information from 


Community CarePlus was received May I5, 2006. 


FINDINGS 

The first PIP evaluated was “Early Intervention in Prenatal Care Management and the 
Relationship to the Very Low Birth Weight Babies.” This project asked if “early identification 
and intervention in prenatal case management decreases the incidence of very low birth weight 
babies?” Very low birth weight babies were defined as weighing less than 1500 grams at birth. 
The steps taken by Community CarePlus included early intervention and implementation of case 
management for all pregnant members. The goal was to increase members’ access to prenatal 
care in an effort to ensure all appropriate health care was received thereby reducing the 
incidence of very low birth weight deliveries. The MCO found that national trends indicated an 
increase in very low birth weight deliveries, so Community CarePlus’s stated goal was for their 


trend data to remain flat. 


The study question was defined. The rational for the study, including the background 
information utilized to support the decision to select this topic, was not identified. Additional 
information on the literature review used to support the choice of a study topic is needed. The 
goal of impacting the trend to “remain flat” did not appear to have a significant impact on the 
identified population. The population was defined to include any MC+ member who was 
pregnant. Members to be included in the study were to be identified by the following methods 
of notification: 
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» Pregnancy Risk Screening Forms 

= Baseline Health Assessment 

= Hospital Admissions and Observations 

= Welcome Calls 

= OB Provider referrals 
The study indicators identified were early identification of members who are pregnant and early 
implementation of case management services. The study did include levels of risk, which 
determined the intensity of case management services. The entire case management 
intervention was well described. The stated decision-making process for determining the level 
of care was “‘clinical and past experience.” This methodology was described in detail providing 
confidence in the decision-making criteria to determine risk. As the study matured appropriate 
changes were made to increase the effectiveness of the interventions. During the first six 
months of 2005 Community CarePlus case managed high risk pregnancies. Throughout the 


remainder of the study year, all pregnancies were case managed. More significant outcomes 


were found as the result of this added component. 


The data to be collected could be inferred from the graphs and tables presented in the study. 
However, the narrative does not describe a data collection plan. There is no explanation of the 
process to be utilized in collecting data nor is there an explanation of the measurement cycles. 
Additional information is needed to define causes and variables that may impact the expected 
outcomes. The information provided does not adequately justify or explain what factors led to 


the interventions chosen and how their effectiveness will be measured. 


The study information did clearly define the sources used to collect data. The narrative 
provided did not specify a systematic method for valid and reliable data collection. Additional 
information should be included to ensure that consistent and accurate data is collected over the 
study period. Pre and post-intervention analysis will be important in determining the 
effectiveness of the intervention over time. The study, even with 2005 statistics, was not 
mature enough to measure the overall effectiveness of the interventions applied. It does appear 


that the study can result in credible and interpretable findings. 


The stated goal of this study was to achieve a “flat” rate of incidences of very low birth weight 
infants. No performance improvement was anticipated. Initially the study did not indicate that 


it would measure any variable factors, such as increased or decreased number of pregnancies, 
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the increased or decreased number of members obtaining case management, or the influence of 
earlier determination of risk for the pregnant women. The improved study that began in June 
2005 does include some of this information. The study is planned to continue throughout 2006 
to ensure that outcomes can be evaluated over time. With additional time to analyze the 
effects of the planned interventions it does appear that the study has a potential for positive 


impact on MC+ Members. This will have to be determined upon completion of the study. 


The second PIP evaluated was “Emergency Room Utilization (Children Receiving Emergency 
Room Services at Cardinal Glennon Hospital).” The focus of this study is unclear because there 
are two stated topics. Originally the study was designed to improve the incidents of emergency 
room visits at one St. Louis area children’s hospital. Later, beginning in October 2005, the study 
was narrowed to address “Emergency Room Utilization for Asthma-Related diagnoses for 
Children 5 — |8 years at Cardinal Glennon Hospital.” The study question that was the focus of 
the second topic is “Does education following emergency room visits for asthma increase the 
member’s quality of health by decreasing emergency medical interventions?” The narrative does 
define why this issue was identified for improvement. It does include the references from the 
MCO’s literature review. In the updated information provided on May I|5, 2006, the rational 
and justification for narrowing this topic, and for choosing one facility for tracking and analysis 
was Clarified in the study narrative. The population was identified in the topic description and 
included all MC+ member children age 5-18 that present to the emergency room for asthma 
related matters. There was no sampling conducted. The study population focused on members 


with special health care needs. 


The hypotheses of the narrowed study are: 


= By educating members following emergency room visits for asthma, Community 
CarePlus believes a member’s quality of life will increase because there is a decrease of 
emergency medical interventions. 
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The study indicators look at the number of emergency room visits per month, and the total 
number of visits per member over a twelve month period. The narrative does not specify an 
outcome relationship and requires more detail. The interventions described in the study 
include: 

# An initial telephone contact following the emergency room visit. 

= Education on medication compliance; follow-up visits with the PCP; benefits of utilizing 

the PCP rather than the emergency room. 

= Monthly follow-up telephone calls to ensure compliance. 
The indicators are designed to provide useful information over time. VVith the focus change for 
this PIP occurring in October 2005, it is difficult to determine the effectiveness of the 
interventions to date. The study did include a detailed methodology for demographic and other 
detailed data to be retrieved. The study explained the sources for this data, although it is 
difficult to determine their reliability. The source for originally identifying members is the daily 
emergency room data sheets from Cardinal Glennon Hospital. The reliability of this 
methodology is unclear. The narrative does not clearly explain why Cardinal Glennon Hospital 
is the focus of the study. It does state that “Cardinal Glennon was the focus of the study being 
the main pediatric hospital where most members seek treatment.” There is no data supporting 
this assertion. The MC+ Medicaid Managed Care region served by Community CarePlus 


included nine counties. 


There is no data presented that supports the claim that Cardinal Glennon Hospital will yield the 
most information due to the volume of children seen there over other hospitals in this region. 
There is also no data to support why one hospital is chosen over an aggregate of all the 


hospitals in the region. 


There was no data analysis plan identified in the narrative. The specific data to be collected was 
not explained. Diagrams and graphs are presented. However, how the information is pertinent 
to the anticipated outcomes is never explained. Data sources are defined, but it is difficult to 
assure that complete and accurate data will be collected. Charts include information on ER 
visits for asthma, how members are referred, times of day that members are seen, and days of 
the week members go to the ER. How any of these are related to the hypotheses or how this 


information relates to anticipated outcomes is not described. 
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lt appears that reasonable interventions are planned, but this is not supported in the details of 
the narrative provided. The second study question was only in place for three months during 
2005. This did not allow for any meaningful analysis of the information presented in the plan. 
With maturity, a detailed explanation of how the data collected will be used, what the expected 
outcomes of the planned interventions are, and how these interventions improved health care 
services for members, this planned study has potential for real and sustained improvement. At 
this time there is not enough information, data, or analysis presented to make any judgment 


about anticipated outcomes. 


STRENGTHS 


|. Although both topics had a clinical focus, they were focused on identifying and correcting 
deficiencies in care or services rather than on utilization of cost. 


2. Both topics included well-stated study questions. 


3. The topics chosen were selected to impact perceived negative outcomes for members. 
One study included an analysis of enrollee needs, background information, and a thorough 
literature review to support the choice. 


4. The responses obtained at the on-site review indicated that the MCO continues 
development of skills in this area and has made the commitment to utilize the PIP process to 
assist in program and organizational development. 


5. Well thought out interventions were planned in both projects. 


AREAS OF IMPROVEMENT 


|. Study questions, measures, and analysis were not related to one another. Quarterly data 
analysis was planned and stated in one PIP. The data gathered and included should have its 
relevance to the stated outcomes explained. 


2. Data analysis should incorporate statistical significance testing to ensure that any resulting 
change, or lack of change, was related to the intervention. 


3. Provide enough narrative to ensure that the reader understands the problem, the proposed 
interventions, the goals and outcomes hoped for, and how the data presented relates to all 
these issues and either supports program improvement, or is not effective. 
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RECOMMENDATIONS 


|. The study design of PIPs needs to link the questions, the interventions, and the proposed 
outcomes to determine whether of not an intervention was effective. This can be 
accomplished by developing a logic model for the PIPs at the planning stage, and ensuring 
that adequate narrative accompanies the data and information presented to make all 
necessary connections. 


2. Quarterly measurement should be utilized if at all possible. This will provide information on 
the ongoing effects of the planned program. 


3. Include a non-clinical PIP in the next planning cycle. 
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6.2 Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Community Care Plus. Community Care Plus submitted the 
requested documents on December 8, 2005. The EQRO reviewed documentation between 
December 9, 2005 and February 27th, 2006. On-site review time was used to conduct follow- 
up questions and provide feedback and recommendations regarding the performance measure 


rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Baseline Assessment Tool (BAT) submitted by Community Care Plus (prepared by Novasys) 

e Healthcare Research Associates’ (HRA) HEDIS 2005 Compliance Audit Report 

e NovaSys Health Network, LLC, policies and procedures related to the HEDIS rate calculation process. 

e —NovaSys Health Network, Community Care Plus electronic eligibility process 

e Data files from the HEDIS repository containing eligible population, numerators and denominators for 
each of the three measures 

e Decision rules & queries in the HEDIS 2005 repository used to identify eligible population, numerators 
and denominators for each of the three measures 

e Query result files from the repository 


The following are the data files submitted by Community Care Plus for review by the EQRO: 


e ADV ENROLLMENT _DATA.txt 

e CIS DEN AND NUM DATA _HYBRID.txt 

e CIS DENOMINATOR_AND_NUM DATA xt 

e CIS ENROLLMENT _DATA.txt 

e =WI5 ENROLLMENT DATA.txt 

e WCV_ DENOMINATOR _AND_NUM_DATA.txt 


Interviews 

The EQRO conducted on-site interviews with Cathy Mocca, Michael Boone (representing 
Novasys, the third party administrator for Community Care Plus) and Ainette Martinez 
(Bridgeport Dental) on Tuesday, February 28, 2006. Michael Boone of Novasys was responsible 


for calculating the HEDIS 2005 performance measures. 
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FINDINGS 

Community Care Plus calculated the Childhood Immunization Status, Combination #2 measure 
using the Hybrid Method. The Well-Child Visits and Annual Dental Visit measures were 
calculated using the Administrative Method. MCO to MCO comparisons of the rates of 
Childhood Immunization Status Combination #2, Well-Child Visits, and Annual Dental Visit 
were conducted using two-tailed z-tests. For comparisons that were statistically significant at 
the 95% confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), 


and the significance levels (p < .05) are reported. 


The rate for the HEDIS 2005 Childhood Immunization Status, Combination #2 measure 
reported to the SMA and the State Public Health Agency (SPHA) by Community Care Plus was 
47.45%. This was significantly higher than the statewide rate for all MC+ MCOs (28.17%; z = 
05; 95% Cl: 36.35%, 58.55%; n.s.). 


The rate for Community Care Plus for the HEDIS 2005 Well-Child Visits measure of 6 or more 
visits was 32.6%; comparable to the statewide rate for all MC+ MCOs (38.42%; 

z= -.62, 95% Cl: 20.07%, 45.13%; n.s.). The 2005 HEDIS combined rate for Annual Dental Visit 
was 31.13%; comparable to the statewide rate for MC+ MCOs (29.76%, z = .22; 95% Cl: 
26.85%, 35.41%; n.s.). 


The following sections summarize the findings of the process for validating each of the 

performance measures in accordance with the Validating Performance Measures Protocol. The 
findings from all review activities are presented according to the EQRO validation activity, with 
the findings for each measure discussed within the activities as appropriate. Please refer to the 


tables in the main report for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

Information systems management policies and procedures for rate calculation were evaluated 
consistent with the Validating Performance Measures Protocol. This included both manual and 
automatic processes of information collection, storing, analyzing and reporting. The EQRO was 
provided with a demonstration of the HEDIS repository. This was done through a remote 
connection from the Community Care Plus location in St. Louis to the vendor’s system in Little 
Rock, Arkansas. For the Childhood Immunization Status, Combination #2, Well-Child Visits and 
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the Annual Dental Visit measures, Community Care Plus was found to meet most of the criteria 
for having procedures in place to produce complete and accurate data (see Attachment V: Data 
Integration and Control Findings). There were no biases or errors found in the manner in 

which Community Care Plus transferred data into the repository used for calculating the HEDIS 


2005 measures. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures were adequate (see Attachment VII: 
Data and Processes Used to Calculate and Report Performance Measures). Community Care 
Plus met nearly all criteria that applied for all three measures. The two criteria that were not 
met involved the use of statistical significance testing to document changes in performance over 
time and using this information to support claims of improvement or stability in performance 
over time. Although many auditors conduct statistical tests to examine various aspects of data 
validity, such tests should be used by the MCO to assess the significance of change related to 
quality improvement activities and operational changes. For the Childhood Immunization Status, 
Combination #2 measure, the State Public Health Immunization Registry (MOHSAIC) data were 
not incorporated into the calculation of the rates, which may have contributed to an 


underestimate of the actual rate. 


Processes Used to Produce Denominators 

Community Care Plus met all criteria for the processes employed to produce the denominators 
of all three performance measures (see Attachment X: Denominator Validation Findings). This 
involved the selection of members eligible for the services being measured. Four hundred 
eleven (411) sampled members were reported and validated for the Childhood Immunization 
Status, Combination #2 measure. A total of |,305 members eligible were reported and | 303 
were validated for the Well-Child Visits measure. Two members were excluded from the 
denominator by the EQRO due to not being enrolled in Community Care Plus on their |5- 
month birthday. A total of 22,621 members eligible were reported and validated for the 
denominator of the Annual Dental Visit measure. The EQRO found the age ranges, dates of 
enrollment, medical events, and continuous enrollment criteria were programmed to include 
only those members who met HEDIS 2005 criteria. Member identification numbers and dates 
of birth were within valid ranges for each of the three measures. The dates of enrollment 


represented valid gaps in enrollment and met continuous enrollment requirements. Medical 
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event codes were also valid for all three measures. There were no exclusions or 
contraindications reported by the MCO. One record was replaced after determining that valid 
data errors occurred; this record was replaced with the next record from the auxiliary list, 


comprised of the final sample size of 453. 


Processes Used to Produce Numerators 

All three measures included the appropriate administrative data ranges for the qualifying events 
(e.g., immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2005 
criteria (see Attachment XIll:_ Numerator Validation Findings) for the Adolescent Immunization 
Status and Use of Appropriate Medications for People with Asthma measures. Medical record 


reviews were conducted for the Childhood Immunization Status measure. 


For the Childhood Immunization Status measure, Community Care Plus excluded administrative 
events from the State Public Health Immunization Registry (MOHSAIC). There were 9 
administrative hits reported and validated by the EQRO. There were 186 hits from medical 
record review reported by Community Care Plus. Thirty (30) of 30 medical records requested 
for review were received and 20 records resulted in validated hybrid hits. The EQRO validated 
133 of the 195 hits reported. The rate validated by the EQRO was 32.36%; this resulted in a 


15.09% estimated bias for the Childhood Immunization Status, Combination # 2 measure. 


For the Well-Child Visits measure, there were a total of 425 administrative hits found in the 
data file; the MCO reported a total of 462 hits. The rate validated by the EQRO was 32.57%; 


the rate reported for this measure was 32.60%, resulting in no bias. 


The Annual Dental Visit was the third measure validated. There were a total of 7,043 
administrative numerator events reported and validated. The diagnosis codes and dates of 
service were valid. The dates of birth were in the valid range. The final rate was calculated to be 


31.13%, with no bias observed. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Childhood Immunization Status measure. Attachment XIl; 
Impact of Medical Record Review Findings and Attachment XV: Sampling Validation Findings 
were completed for each of these measures. Community Care Plus employed a 10% 
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oversampling rate for a final sample size (FSS) of 453, which is within specified parameters. | 


(one) replacement was reported from administrative data. 


Submission of Measures to the State 

Community Care Plus submitted the DST for each of the three measures validated to the SPHA 
(the Missouri Department of Health and Senior Services; DHSS) in accordance with the Code of 
State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the 


SMA Quality Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 
The following table summarizes the estimates of bias and the direction of the bias. There was 


no bias found for both the Well-Child Visits measure and the Annual Dental Visit measure. 


Table 52 - Estimate of Bias in Reporting of HEDIS 2005 Measures 


Measure Estimate of Bias Direction of Estimate | 


Childhood Immunization Status, Combination #2 Overestimate 
Well-Child Visits in first 15 months of Life 
Annual Dental Visit 





Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all 
data sources summarized in the Final Performance Measure Validation Worksheet for each 
measure. Table 53 (see below) shows that the Childhood Immunization Status, Combination #2 
measure was not valid due to the validated rate falling below the 95% lower confidence limit 


reported by Community Care Plus. 
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Table 53 - Final Audit Rating for Performance Measures 

Measure Final Audit Rating 

Childhood Immunization Status, Combination #2 Not Valid 

Well-Child Visits in first 15 months of Life Fully Compliant 

Annual Dental Visit Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by the MCO. Not Valid = Measure deviated from State specifications such 
that the reported rate was significantly biased. This designation is also assigned to measures for which no rate was 
reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Calculation of the HEDIS 2005 Annual Dental Visit measure was fully compliant with 
specifications. 


2. Community Care Plus collected and submitted 30 of the 30 medical records requested for 
the Performance Measure validation. 


3. The data repository was very well-designed and had methodically arranged queries. Most of 
the queries were ‘make table’ queries, which received the data online from the Amisys 
database. The data is then ‘frozen’ into the HEDIS repository built-in MS Access. The 
source code was reviewed for accuracy and the results were verified. 


4. The arrangement of the tables and queries was step-by-step and top-down, which facilitated 
a detailed overview of the process and it minimized the possibility of error. An example of 
a step-by-step methodology was having a separate member span table, which was updated as 
soon as the MCO received changes in members’ status. This table calculated continuous 
enrollment in a detailed fashion and also ruled out gaps in enrollment. The EQRO 
conducted a validation test on the Dental measure. Standard codes such as ICD-9 and UB- 
92 were checked for accuracy and updated versions. 


5. Up-to-date code versions were provided to Novasys by their auditor HRA, and were well- 
integrated into the system, producing reliable results. 


6. There was excellent data integration with automated quality checks and reports. Data was 
collected from Amisys, the claims management system for membership, enrollment and 
encounter data. Pharmacy data was merged from ExpressScripts via Amisys into the 
warehouse. Controls for data checking and error handling were present within the Amisys 
system. 


7. The data resided in a secure location within Perot Systems, the owners of Amisys, (located 
in Dallas, Texas). The data was transferred through a secure ODBC connection. 


8. Information system policies were present within Novasys to maintain the integrity of the 
data. 


9. Workflow and documentation of the working mechanisms of the HEDIS repository (used to 
provide external observers and subsequent users with data field definitions) is greatly 
improved since last year’s audit. 
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AREAS FOR IMPROVEMENT 


|. Data from external databases, such as the State Public Health Immunization Registry 
(MOHSAIC), for collection of childhood immunization data, was excluded from the rate 
calculation. The current rates are likely significantly underestimated as a result of omitting 
MOHSAIC data. 


2. Community Care Plus has indicated a need to upgrade from an Access system of HEDIS 
rate calculation to an SQL system; the EQRO sees this as an area that should be pursued. 


3. Continue to encourage electronic claim submissions from providers as much as possible. 


RECOMMENDATIONS 


|. Data from MOHSAIC for the Childhood Immunization Status measure should be 
incorporated into measure calculation to produce rates in a reliable manner for comparison 
with other MC+ MCOs. This can be done by developing a data exchange procedure with 
the SPHA. Once a year, MCOs send a list of eligible members to the SPHA, which will 
return immunization data for the members. These procedures were outlined for the MCO 
in April, 2005. During the site visit on March 6, 2006, the MCO indicated that they have 
worked with the SPHA to participate in the data exchange for the 2006 HEDIS season. 


2. Continue the move to a structured data warehouse, such as SQL Server from the current 
MS Access repository, as it will facilitate better data retrieval and analysis. 


3. Continue work with MEDLine to improve the number of electronic claims submitted by 
providers; this should improve accuracy and timeliness of claims. 
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6.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical Fields? 


For the Medical claim type, there were 77,055 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 


The Outpatient Claim Type field was |100.00% complete, accurate and valid. 
The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 


The Outpatient First Date of Service field was |00.00% complete and accurate, and 99.93% valid. 
Invalid dates ranged from 12/03/2005 — 12/30/2005. 


The Outpatient Last Date of Service field was |00.00% complete and accurate, and 99.91% valid. 
Invalid dates of service ranged from 04/01/2005 — 04/29/2005. 


The Outpatient Units of Service field was |00.00% complete, accurate and valid. 

The Outpatient Procedure Code field was 100.00% complete, accurate, and valid 
The Outpatient Place of Service field was 100.00% complete, accurate and valid. 

The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA 
for this validation. The second Diagnosis Code field was 31.22% complete, accurate and 
valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The third 
through fifth Diagnosis Code fields were 0.00% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and 
invalid). 


For the Dental claim type, there were 5,225 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate 


and valid. 


For the Home Health claim type, there was one (1) encounter claim paid by the SMA for the 
period January |, 2005 through March 31, 2005. All fields examined were 100.00% with the 
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following exceptions. The Procedure Code and the fourth and fifth Diagnosis Code fields were 


blank. 


For the Inpatient claim type, there were 2,255 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 
The Admission Type field was 100.00% complete, accurate and valid. 


The Admission Date field was 100.00% complete and accurate, and 98.18% valid. There were 
4| invalid dates ranging from 12/10/2005 — 12/31/2005. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 97.74% (with 52 entries of 
99999999"). Valid values were present 89.31% of the time. In addition to the invalid 
“99999999” entries, 189 invalid dates ranged from 04/01/2005 — 04/15/2005. 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 
The first Diagnosis Code field was 100.00% complete, accurate and valid. 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (93.48%, 82.17%, 61.46%, 
41.73%, respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


The First Date of Service field was 100.00% complete and accurate, and 98.18% valid. There 
were 41 invalid dates ranging from 12/10/2005 — 12/31/2005. 


The Last Date of Service field was 100.00% complete and accurate, and 91.40% valid. The 193 
invalid dates of service ranged from 04/01/2005 — 04/29/2005. 


The Revenue Code field was 100.00% complete, accurate and valid. 


The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were 45,/82 encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. 


The Outpatient Claim Type field was |00.00% complete, accurate and valid. 

The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete and accurate, and valid. 

The Last Date of Service field was 100.00% complete and accurate, and valid. 

The Units of Service field was 100.00% complete, accurate and valid. 

The Outpatient Procedure Code field was 100.00% complete and accurate, and 77.43% valid. 
There were 10,332 entries of the invalid code “00000”’. 
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The Revenue Code field was 100.00% complete, accurate and valid. 
The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA 
for this validation. The second Diagnosis Code field was 84.68% complete, accurate and 
valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The third 
Diagnosis Code field was 40.82% complete, accurate and valid (incomplete, inaccurate, and 
invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was | 7.70% complete, accurate and valid. The remaining fields were 
blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code field was 7.23% 
complete, accurate and valid. The remaining fields were blank (incomplete, inaccurate, and 
invalid). 


For the Pharmacy claim type, there were 78,518 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Community Care Plus, an error analysis of the invalid entries 
was conducted for fields which were lower than the |00.00% threshold specified by the SMA. 
There were very few errors encountered in the critical fields examined across all claim types. 
The Hospital Outpatient Procedure Code field contained a large proportion of invalid entries. 
For the Inpatient claim type, the Discharge Date field contained invalid entries, but this may be a 
function of patients who were admitted during the specified time period but were not yet 


discharged within the period. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, Community Care Plus 
demonstrated significantly lower rates than the average for all MC+ MCOs for the Medical, 
Dental, and Outpatient Hospital claim types. This may be a function of provider panel 
composition or claims administration. The possibility of incomplete data cannot be ruled out 
given the consistent pattern of low rates across claim types. Another possible explanation is 


less access to care for members, or a healthier member population. 
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To What Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 
types for the period of January |, 2005 through March 31, 2005 for medical record review. Of 
the 77,055 Medical encounter claim types in the SMA extract file for January |, 2005 through 
March 31, 2005, 100 encounters were randomly selected. Providers were requested to submit 
medical records for review. There were 77 medical records (77.0%) submitted for review. 
Encounters for which no documentation was submitted were unable to be validated. The match 
rate for procedures was 42.0%, with a fault rate of 58.0%. The match rate for diagnoses was 


71.0%, with a 29.0% fault rate. 


What Types of Errors Were Noted? 
An error analysis of the errors found in the medical record review for procedure and diagnosis 
codes was conducted. For the diagnosis code in the medical record, the reasons for diagnosis 


codes not matching the SMA extract file were missing or illegible information (n = 29). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = | land incorrect codes (n = 47). Examples of missing information included no 
code, codes listed that were not supported, or codes that did not match the procedure 


description. 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing Community Care Plus (CCP) 
encounter data to the SMA encounter claim extract file because there was not an identical 
encounter number to key the two files. The plan was to compare MC+ MCO paid and unpaid 
claims to the SMA encounter claim extract file. While CCP did submit the data in the requested 
format (see Appendix 6) for the MC+ Managed Care Members represented in the encounter 
claim sample selected by the EQRO for validation, there was no way to match the encounters 


without an internal control number (ICN) that is identical to that of the SMA. 
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What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While the MC+ MCO did submit the data in the requested format, there are a number of ways 
to improve the data quality by improving the database system. One variable that is not 
currently represented is that of a unique line number. When an ICN number is submitted, it is 
essentially a visit number, but the numbers submitted by Community CarePlus were not the 
same as the ICN submitted with the state encounter data. To match up specific lines of data 
(each service of the encounter) this would require a unique number for each service provided 


for each member. 


STRENGTHS 


|. Data was submitted in the requested format for encounter validation and all claim types 
were accessed. 


2. The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


3. The critical fields evaluated for the Dental, Outpatient Home Health, and Pharmacy claim 
types were 100.00% complete, accurate, and valid. 


4. Community Care Plus demonstrated significantly higher rates of validation of the SMA 
encounter claims extract file against the medical record for the diagnosis relative to the 
average for all MC+ MCOs. 


5. Claim Status (Paid, Denied, & Unpaid) was submitted. 


6. Revenue Codes were 100% complete, accurate, and valid for both Inpatient and 
Outpatient Hospital claim types which were incomplete last year. 


AREAS FOR IMPROVEMENT 


|. For the Medical claim type, the Outpatient First Date of Service and Outpatient Last Date 
of Service fields contained some invalid entries, resulting in a valid rate of 99.93% and 
99.91%, respectively. 


2. For the Inpatient claim type, the Admission Date and the Discharge date fields contained 
invalid entries, resulting in a valid rate of 98.18% and 89.31, respectively. 


3. The Outpatient Hospital claim type had invalid codes in the Outpatient Procedure Code 
field. 


4. Community Care Plus demonstrated significantly lower rates than the average for all MC+ 
MCOs for the Medical, Dental, and Outpatient Hospital claim types. 


5. Community Care Plus demonstrated significantly lower rates of validation of the SMA 
encounter claims extract file against the medical record for the procedure codes relative 
to the average for all MC+ MCOs. 
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RECOMMENDATIONS 


The MC+ MCO should examine the rate of claims per |,000 members across claim types 

and the rate of rejected claims for each claim submission format (UB-92, NSF/CMS 1500, 

NCPDP 3.0) over time to examine the consistency in claims submission and identify issues 
for data submission. The access to care should also be examined as a possible reason for 

the lower rates of encounter claims per |,000 members. 


Examine and revise as needed internal system edits for invalid procedure codes in the 
NSF/CMS 1500 file layout and run validity checks after the programming of new edits. 


For the Inpatient claim type (UB-92 file layout), improve the rate of valid Discharge Dates 
to flag invalid entries of “99999999”. 
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6.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract documents with the staff of the Division of Medical Services. On-site review 
time was used to conduct interviews with those who oversee the daily practices of the MCO. 
This ensures that documentation is developed and practices occur within the scope of the 


contract and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The following documents pertaining to Community CarePlus were reviewed prior to and at the 
on-site visit: 


The Division of Medical Services supplied: 
» State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

= Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection process of actions filed in the first quarter of 2005. 

# 2004 Annual Quality Improvement Annual Summary 
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Additional documentation made available by Community CarePlus included: 
# 2005 and 2006 Marketing Plan 
# Organizational Chart 
# 2005 PIP ER Utilization for Asthma 
= 2005 PIP Early Prenatal Intervention 
# 2005 Call Center Foreign Language Reports 
= CCP Welcome Packets, with correspondence, postcards, privacy/HPPPA information 
= Healthy Moves 2005 CCP Member Newsletters 
# 2005 Quality Monitoring Log 
= CCP Policy Tracking Log 
» Unity Managed Mental Health — QI Workplan 2005 — Outcomes 
» Unity Managed Mental health — CCP 2005 Program Reports 


Interviews 
Interviews were conducted with the following group: 
Plan Administration 
Jerry Linder — President and CEO 
Robert Profumo, MD — Medical Director 
Cris Cristea — Chief Operating Officer 
Beverly Thompson — Director, Medical Management 
Marcia Albridge — Director, Business Development 


Kathy Mocca — Manager, Business Development 
Robin Woolfolk — Manager, Customer Service 


Mental Health 
Marcia Albridge — CCP 


Unity Health Services: 
Scott Frederick, Director, Managed Mental Health 
Marjorie Viehland, Manager, Utilization Management and Quality Improvement 


FINDINGS 

Enrollee Rights and Protections 

Community CarePlus made a serious effort to begin tracking and monitoring all policy required 
to be submitted to and reviewed by the SMA. This included policy and procedures for initial as 
well as annual approval materials. Additionally, the MCO developed an inventory of all written 
materials or purchased materials that must be approved by the SMA prior to being shared with 
members. A binder including all Annual Marketing Materials was compiled and shared at the on- 
site review. It was observed that the MCO made necessary changes that indicated their 


current address and telephone numbers for members to reference when they contact 
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Community CarePlus. The MCO stated that after their move in 2004, handout information was 


incorrect, but the primary toll-free contact number, accessed by members, remained the same. 


The Member Handbook was approved by the SMA and is now in a recorded format to be 
shared with members who are visually impaired or have other challenges with written material. 
Certified interpreters for deaf or non-English speaking members are provided as needed. The 
International Institute and the Language Access Metro Project (LAMP) are the primary 
resources used for interpretive services by Community CarePlus. The MCO reported receiving 
an average of fifty-nine (59) calls per month requiring interpretive services. 

Training is regularly provided to ensure that the Community CarePlus Call Center staff is 
knowledgeable about members’ rights and responsibilities. All incoming calls are monitored and 
additional in-service training and coaching is provided based on information gathered. Call 
Center staff rotated to provide 24-hour coverage on holidays and weekends. They also assist in 
contacting new members. The MCO began utilizing internet resources, as well as the new 
member report received from the SMA, to have up-to-date information on members. In late 


2005, the MCO reached a 60% success rate for contacting new members. 


Community CarePlus continues to enhance case management services to members with special 
needs. They review all sources to identify members in need of case management, and provide 
them with individual attention as quickly as possible. Case managers track members who are 
pregnant, have high blood lead levels, have identified special healthcare needs, and any 
catastrophic illness. The MCO reported that they follow about twenty-five (25) members 
through case management services at a time. The Lead Case Manager maintains a database with 
information provided by the SMA, and is active in educating providers regarding the use of 
capillary testing to encourage blood lead level tests for children. The medical director met daily 
with case management staff to discuss cases and held weekly case conferences. This type of 


support was beneficial to the MCO and to case management activities. 


The rating for Enrollee Rights and Protections (46.2%) indicates that Community CarePlus 
continued to have written policies and procedures that were not submitted to the SMA for 
review and approval. However, it should be noted that the MCO made a significant 
improvement in this area, has improved tracking and internal processes, and is in the process of 


completing policy development and submission. Community CarePlus exhibited a business like 
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approach and commitment to continue their efforts to improve in the completion and 
submission of required policies and procedures. Their stated goal was to become fully 


compliant with all MC+ Medicaid Managed Care contract requirements and federal regulations. 


Table 54 - Subpart C: Enrollee Rights and Protections Yearly Comparison 


Federal Regulation 





438.100(a) Enrollee Rights: General Rule 1 2 
438.10(b) Enrollee Rights: Information Requirements 0 1 
438.10(c)(3) Alternative Language: Prevalent Language 1 2 
438.10(c)(4,5) Language and Format: Interpreter Services 1 1 
438.10(d)(1)(i) Information Requirements: Format/Easily 

1 2 
Understood 
438.10(d)(1)(ii)and (2) Information Requirements: Format Visually 0 1 
Impaired, and Limited Reading Proficiency 
438.10(f) Information for All Enrollees: Free Choice, etc. 1 1 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 5 5 
Plans 
438.10(i) Special Rules: Liability for Payment/Cost Sharing 1 1 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee Communications 1 2 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 1 1 
438.100(b)(3) Right to Services 1 1 
438.100(d) Compliance with Other Federal/State Laws Ps 2 
Number Met 2 6 
Number Partially Met 9 7 
Number Not Met 2 0 


Rate Met 15.4% 46.2% 
Note: 0 = Not Met; 1= Partially Met; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Behavioral Health 

Unity Managed Mental Health (UMMH) is the Behavioral Health Organization (BHO) that 
subcontracts with Community CarePlus for mental and behavioral health services for members. 
The BHO presented and discussed their 2005 program reports. The reports included data on 
inpatient and outpatient utilization, re-admission within thirty (30) days, frequency of diagnosis, 
inpatient age distribution, member and provider satisfaction, primary care physician coordination 
of care, pregnant women’s care coordination, and provider response to clinical guidelines. 
UMMH provided additional information on interpreting this data and the content of their 2005 


activities. 
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UMMH developed and utilized clinical guidelines in the areas of depression and ADHD. The 
BHO continued work on guidelines for Pregnant Women’s Care Coordination. The MCO was 
involved in this project. Unity Managed Mental Health found that 22% of identified pregnant 
members accessed mental health services. Their report indicated that their coordinated case 
finding efforts improved by 188% during 2005. The majority of these cases were identified by 
Community CarePlus case managers. The BHO and MCO expressed a continued commitment 
to developing guidelines and practice in the area of providing appropriate mental health support 


to pregnant members. Their goal is to prevent negative outcomes for the member or infant. 


Case management and pain management were additional areas that the BHO was working on 


with the MCO. 


Quality Assessment and Performance Improvement 


Access Standards 

All Community CarePlus prior authorization policy was approved during 2005. The MCO had a 
schedule to submit policies and procedures to the SMA for annual review as required. The 
MCO explained that currently all authorizations were received from providers telephonically. 
Community CarePlus staff measured the requests and accompanying information against 
InterQual criteria. If the decision was to deny the authorization, the information was reviewed 
by the medical director prior to entry into the MCO system. All authorizations were tracked 
and monitored. The MCO required prior authorization of all inpatient stays, MRI, CT scan, 
physical therapy, occupational therapy, speech therapy, certain medications, home health 
services and pain management. Community CarePlus made it clear that there were no prior 
authorizations required for emergency services, and that all emergency services are reimbursed 


regardless of who provided them. 


A new process was implemented during 2005 to track authorization denials. Community 
CarePlus decreased the timeframes for responding to authorization requests. They updated 
their policy to ensure that denials would be overturned when adequate information was 


provided. Tracking and trending of information occurred and was reviewed on a monthly basis. 


Some areas of provider access improved during 2005. The MCO added St. Anthony’s hospital, 


which the MCO believed would positively impact availability of emergency services for Franklin 
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County members. Community CarePlus admitted that this hospital was not actually located in 
Franklin County, and that it would be over 50 miles for many residents to get to this hospital. 
However, they made assurances that St. John’s Hospital in Washington and Missouri Baptist 
Hospital in Sullivan would see MCO members for emergencies or after hours care. Community 
CarePlus reported that although the Barnes-Jewish-Christian Hospital System and St. John’s 
Mercy Hospitals were not in their network, they maintained a strong relationship with these 
systems. These hospitals are utilized as out-of-network providers. Community CarePlus was 
able to add the Barnes-Jewish-Christian (BJC) and Washington University Physicians groups to 


their provider panel during the past year. 


Community CarePlus reported that they continue to struggle with finding several specialty 
providers, particularly pediatric neurologists and orthopedic surgeons. The MCO had been able 
to negotiate for these services because the Provider Relations staff developed individualized 
relationships with providers. They did report paying orthopedic surgeons 100% of billed 


charges. 


Community CarePlus assessed provider availability annually when producing their report to the 
Missouri Department of Insurance. In 2005 the MCO conducted a study on the availability of 
24-hour coverage by providers, as required in their MC+ Medicaid Managed Care Contract. 
The MCO monitored provider telephone logs, conducted blind telephone testing, and obtained 
input from providers directly. Community CarePlus reported doing well in this area, but 
admitted that there were providers who needed work and continued testing. The MCO 
reported that they contracted with all of the Federally Qualified Health Centers (FQHCs) in 


their MC+ Managed Medicaid area that improved daytime and some after-hours access. 


Community CarePlus revised their access to service policy to ensure that urgent services were 
available within twenty-four (24) hours. The original policy stated that urgent care was required 


within 48 hours. 


Ratings for compliance with Access Standards (47.1%) reflect a serious attempt by the MCO to 
complete required policy to meet the requirements of the MC+ Medicaid Managed Care 
contract and federal regulations. However, all required policy and procedure are not complete. 


Community CarePlus must continue their efforts to develop necessary policy and practice to be 
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in full compliance. Observations made at the time of the on-site review indicated that these 


efforts were underway. 


Table 55 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 


CCP 


Federal Regulation 





438.206(b)(1)(i-v) Availability of Services: Provider Network 
438.206 (b) (2) Access to Well Woman Care: Direct Access 


438.206(b)(3) Second Opinions 


438.206(b)(4) Out of Network Services: Adequate and Timely 
Coverage 


438.206(b)(5) Out of Network Services: Cost Sharing 
438.206(c)(1)(i-vi) Timely Access 

438.206(c)(2) Provider Services: Cultural Competency 
438.208(b) Care Coordination: Primary Care 

438.208(c)(1) Care Coordination: !dentification 
438.208(c)(2) Care Coordination: Assessment 

438.208(c)(3) Care Coordination: Treatment Plans 
438.208(c)(4) Care Coordination: Direct Access to Specialists 
438.210(b) Authorization of Services 

438.210(c) Notice of Adverse Action 


BSN ey N 
eS N ey N 


438.210(d) Timeframes for Decisions, Expedited Authorizations 
438.210(e) Compensation of Utilization Management Activities 


438.114 Emergency and Post-Stabilization Services 
Number Met 
Number Partially Met 
Number Not Met 3 
Rate Met 23.5% 47.1% 
Note: 0 = Not Met; 1= Partially Met; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Structures and Operation Standards 

Community CarePlus continued to develop their credentialing standards. The MCO assured 
that providers maintained licensure and the right to practice in Missouri. Source One was 
employed to run a monthly data scan against licensing listings. This process enabled the MCO 
to maintain current licensure information. Community CarePlus reported that they were 
current on all credentialing for new physicians and on delegated credentialing. The MCO 


developed a work plan to ensure that the remaining provider list would be current during the 
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coming year. The MCO expected to be current on all providers due for credentialing within a 


few months. 


In the fall of 2005, an after-hours survey was conducted by telephone to assess twenty-four 
hour primary care physician (PCP) availability. Community CarePlus worked toward making 
after-hours services available to prevent the unnecessary use of emergency rooms. If providers 
did not meet the requirements for after-hours availability, they received education on policy. 
Provider representatives visited these PCP offices every six weeks for follow-up, and provided 
additional assistance to trouble shoot specific issues. Community CarePlus developed a 


definition of “medical necessity” as required. 


The rating for Structure and Operation Standards (80%) reflects the timely and efficient 
submission of policy to the SMA for their review and approval. The MCO understood that 
continued improvement in this area of practice was needed. However, they continued to make 
progress. Observations at the time of the on-site review supported Community CarePlus 


success at identifying and improving areas that had previously been problematic. 


Gea Performance Management Solutions Group 
eee A division of Behavioral Health Concepts, Inc. 213 


Missouri MC+ Managed Care External Quality Review Section 6 
Report of Findings — 2005 Community CarePlus 


Table 56 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


CCP 


Federal Regulation 





438.214(a,b) Provider Selection: Credentialing/Recredentialing 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 
438.214(d) Provider Selection: Excluded Providers 


438.214(e) Provider Selection: State Requirements 


438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 
limitations 


438.56(c) Disenrollment Requested by the Enrollee 
438.56(d) Disenrollment: Procedures 

438.56(e) Disenrollment: Timeframes 

438.228 Grievance System 

438.230(a,b) Subcontractual Relationships and Delegation 


Number Met 
Number Partially Met 
Number Not Met 
Rate Met 40.0% 80.0% 
Note: 0 = Not Met; 1= Partially Met; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Measurement and Improvement 

Community CarePlus had not developed or implemented specific practice guidelines with 
providers. The MCO had a new medical director and planned to confront this issue. Concern 
was expressed about resistance to the use of practice guidelines in the St. Louis area. The 
requirements of the federal regulations regarding practice guidelines were shared with 
Community CarePlus. The medical director considered decisions based on “what is appropriate 
for the member and supports care that is based on peer review and outcome based research,” 
as a sound benchmark. The need to comply with the requirements in the federal regulations 


was stressed to the MCO. 


Community CarePlus instituted a number of Quality Assessment and Performance Improvement 
activities during 2005. Their Quality Improvement group met quarterly and included five 
physicians who actively participated. The MCO’s goal of providing quality services to members 
was the focus of the group’s discussions. Community CarePlus viewed this initiative as having a 


positive effect on the performance and focus of the MCO. A suggestion generated by this group 
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led the MCO to hand deliver EPSDT lists to providers. By making personal contact, the 
importance of performing EPSDT exams and providing feedback to the MCO was stressed. 
Community CarePlus worked with the Missouri Department of Health and Senior Services 
(DHSS) to obtain immunization information since November 2004. The MCO continues its 


attempts to obtain access to the DHSS MOSAIC system. 


Community CarePlus reported that a major change during 2005 in the Quality Assessment and 
Improvement program was the development and completion of policy regarding member 
Grievance and Appeals, and provider Complaints, Grievances and Appeals. The MCO set up an 
internal monitoring process and found a 100% success rate in sending letters according to policy 


during the first quarter of 2006. 


Community CarePlus submitted two Performance Improvement Projects (PIPs) for validation. 
Although these PIPs lacked maturity to allow for complete validation, they indicated substantial 
improvement in utilization of this process as a tool for MCO growth. The structure of both 


PIPs followed the federal protocol and showed a great deal of potential. 


The MCO submitted all required information to complete the Validation of Performance 
Measures, as requested. Community CarePlus continued to operate a health information 
system within the guidelines of that protocol. All encounter data requested was provided in the 
correct format. The details of each of these areas of validation can be reviewed within specific 


sections of this report. 


The rating for Measurement and Improvement (54.5%) reflects a diligence toward meeting the 
requirements of the MC+ Medicaid Managed Care contract and federal regulations. Many 
policies and procedures have been submitted to the SMA for their review and approval. The 
MCO needed to work on the development, dissemination, and application of clinical practice 


guidelines. Some policy development must still occur. 
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Table 57 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison. 


CCP 


‘Federal Regulation 





438.236(b)(1-4) Practice Guidelines: Adoption 
438.236(c) Practice Guidelines: Dissemination 
438.236(d) Practice Guidelines: Application 


438.240(a)(1) QAPI: General Rules 


438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 
Improvement and PIPs 


438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 


438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 


438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare 
Needs 


438.240(e) QAPI: Program Review by State 

438.242(a) Health Information Systems 2 

438.242(b)(1,2) Health Information Systems: Basic Elements 1 

438.242(b)(3) Health Information Systems: Basic Elements 1 
2 
9 


SP RP Be FP PN BS 
NO NN FY BP YP BB 


= 
> 
= 
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Number Met 

Number Partially Met 

Number Not Met 0 
Rate Met 18.2% 54.5% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; 1= Partially Met; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Grievance Systems 

Community CarePlus completed and submitted the majority of policy and procedures to make 
their Grievance System compliant with MC+ Medicaid Managed Care contract requirements and 
federal regulations. The MCO put processes in place to capture member and provider contacts. 
Although they experienced some reduction in staff, the MCO was working smarter and 
developed better communication between internal departments. This enhanced their ability to 
track and respond to member grievance and appeals, as well as provider complaints, grievances, 


and appeals. 


Two member grievances were provided for review. Copies of the phone log were provided 


rather than the requested member files. Required correspondence was not included. Both 
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grievances related to difficulty accessing providers. In one case, a provider refused to see a child 
because the parent did not speak English. Provider Services staff met with the provider 
regarding their contractual requirements to serve MC+ Medicaid Managed Care members. 

They also provided education about requesting interpretive services when necessary. According 
to the information presented, the provider staff did not recall this incident. In the second 
grievance the member requested a specific provider. The MCO referred the member to their 
open panel protocol and offered two alternative physicians. The member explained that these 
physicians were too far from their home, and asked for a provider in the home community. No 


specific resolution was identified in the information provided. 


Five provider complaints were reviewed. Three of these concerned claim denials as the result 
of late filing. Two were overturned and one was upheld. The fourth complaint requested an 
additional payment and the denial was upheld. These files did contain required information and 
correspondence. In one case the correspondence was mailed outside of the required 
timeframes. The final complaint involved a denial due to lack of authorization for inpatient 
psychiatric services. The case was referred to UMMH for a second level appeal. No record 


was found and no explanation was available. 


Community CarePlus admitted that early in 2005, the time period from which the grievances 
and complaints were requested, their system was in disarray. They did voluntarily produce 
member and provider files from later in the year to exemplify the improvements the MCO 
made in their Grievance System during 2005. These records did contain correspondence and 
documentation that was in chronological order and were more complete. The MCO shared the 
current Grievance System policy and tracking information. It did appear that significant 


improvement occurred in their processes. 


The rating for the Grievance System (94.4%) reflects approval of the majority of policy and 
procedures required to meet MC+ Medicaid Managed Care contract requirements and federal 
policy. Practices observed at the time of the on-site review indicated that currently Community 
CarePlus was meeting all requirements of operating a functional Grievance System for both 


providers and members. Follow-up review of the Grievance System should occur in the future. 
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Table 58 - Subpart F: Grievance Systems Yearly Comparison 


Federal Regulation 





438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2. 
438.404(a) Grievance System: Notice of Action - Language and 1 5 
Format 

438.404(b) Notice of Action: Content 2 
438.404(c) Notice of Action: Timing 1 2 
438.406(a) Handling of Grievances and Appeals: General 1 5 
Requirements 

438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 

438.408(a) Resolution and Notification: Basic Rule 2 

438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 

438.408(d)(e) Resolution and Notification: Grievance and Appeals - 1 5 
Format and Content of Notice 

438.408(f) Resolution and Notification: Grievances and Appeals - l 5 
Requirements for State Fair Hearings 

438.410 Expedited Resolution of Appeals 1 2 
438.414 Information about the Grievance System to Providers and 1 5 
Subcontractors 

438.416 Recordkeeping and Reporting Requirements 0 1 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pends 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 9 17 
Number Partially Met 8 1 
Number Not Met 1 0 


Rate Met 50.0% 94.4% 
Note: 0 = Not Met; 1= Partially Met; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 
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Summary 

Although Community CarePlus was not 100% compliant in any area, it must be noted that the 
MCO made significant improvement in policy and procedure submission and approval in all 
areas. At the time of the 2004 EQR areas of practice not meeting compliance standards were 
also identified. At the time of this review improvement in many areas of performance were 
observed. Community CarePlus continued their commitment to members and to providing 
healthcare services in an effective manner by demonstrating an atmosphere of respect and 


dignity toward members. 


During the on-site review Community CarePlus indicated that they recognized the need to 
improve in the development of policies and procedures, and continue to review and upgrade 
their organization’s performance. They exhibited a commitment to these goals, and provided 
sound examples of the progress made during 2005. Although the MCO was not fully compliant, 
the improvements witnessed at this review provided a sound foundation for continued efforts to 


make the changes required to meet full compliance in the future. 


STRENGTHS 


|. Community CarePlus had specific tracking mechanisms in place to ensure that policy review, 
updates, and submissions occurred in a timely manner. There was evidence of attention to 
the detail required to have accurate and responsive policy. 

2. Community CarePlus exhibited a great deal of concern and commitment regarding their 
responsibility in meeting the state contract requirements and federal regulations. 

3. In planning for the site visit and the 2005 External Quality Review, Community CarePlus was 
responsive to all information and documentation requests. Additionally, the EQRO staff 
witnessed excellent follow-up, tracking, and submission for required information. 

4. There was a distinct recognition within the organization of the need to understand and 
communicate throughout all departments. In conducting the interviews, staff exhibited a 
much firmer knowledge of the business of the MCO. 

5. Continued partnership with Unity Managed Mental Health to meet the need of Community 
CarePlus members. 
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AREAS FOR IMPROVEMENT 


Continue to utilize the resources at Community CarePlus to complete all necessary policy 
documentation and submission to the SMA. 


2. Finalize all improvements and corrections in the Member and Provider Handbooks to 
ensure timely approval, which is required annually. 

3. Development and utilization of established practice guidelines. 

RECOMMENDATIONS 

|. Continue to develop the atmosphere within Community CarePlus that motivates the 
attention to compliance with contractual requirements and federal regulations. A great deal 
of improvement was witnessed in this area. Maintaining these improvements is an important 
factor in continuing the confidence in the MCO. 

2. Continue to utilize available data and member information in order to drive change and 
measure performance. 

3. Continue to enhance the member grievance and appeal system, and the provider complaint, 
grievance and appeal system. The files reviewed at the request of the MCO from late 2005 
were more complete and appeared to follow policy. Processes have been developed to 
internally capture member and provider contacts. Ensure that these processes are followed 
and continue to improve. 
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79. (2MeRCY 


Aealth Phares 


7.1 Performance Improvement Projects 


The previous sections of the 2005 EQRO report present the purpose and objectives, technical 
methods, procedures for evaluation, and MCO to MCO comparisons of analyses, findings, and 
recommendations for each of the protocols based on the aggregate analysis. This section of the 
EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed 
Care Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


METHODS 
Document Review 
Mercy MC+ supplied the following documentation for review: 
#" NCQA Quality Improvement Activity Form: Medicaid Emergency Room Utilization 
# NCQA Quality Improvement Activity Form: Customer Service — Member Service Call 
Center Quality 
Additional documentation was provided at or after the on-site review, and was considered in 
completing the validation, including: 


# Policy MBS0040 — Member Demographic Verification 

# Policy ERO2 — Mercy On Call (Nurse Triage Service) 

# Policy EROI — Conducting Member Outreach Related to Emergency Room Utilization 
# 2005 Consumer Assessment of Health plans Survey 

# Policy HR5015 — Quality Monitoring Process 


Interviews 

Interviews were conducted with the project leaders for each Performance Improvement Project 
(PIP) by the EQRO team on March 2, 2006 during the on-site review, and included the 
following: 


Pat Snodgrass — Director, Performance Improvement 
Anna Dmuchovsky — Chief Medical Officer 

Steve Mead — Product Manager, Medicaid 

Sam Fenner — MC+ Administrator 
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During the interviews, participants shared information and reviewed validation methods, study 
design, and findings. Technical assistance regarding study design and measures was provided by 
the EQRO. The following questions were addressed for specific performance improvement 
projects on-site: 


Medicaid Emergency Room Utilization 


=» Who is the actual study population? 

= How is this topic related to quality, access, and timeliness of health care services? 
=» What are the stated indicators related to? 

=» What methods will be used to collect valid and reliable data? 

= How are the proposed data measures related to the desired outcomes? 
Customer Service — Member Service Call Center 


= How is customer satisfaction tied to the proposed intervention? 

» What is the study question for this PIP? 

#" How will proposed indicators be defined or measured? 

# What follow-up measures will be utilized to identify improvement? 

= Specify what exact data sources will be analyzed to identify improvement. 

s» What interventions took place and how do they relate to the outcomes defining 
improved member services? 


FINDINGS 

The first PIP evaluated was “Medicaid Emergency Room Utilization.” The project narrative 
stated the MCO believed that “excessive time in the emergency room affects one’s quality of 
life. “ Members targeted were those that had three or more emergency room visits for a low 
severity problem in a twelve-month period, and no physician visits during the same time period. 
A literature review and analysis of MCO and regional statistics was conducted to provide 
evidence of the effects of this type of member behavior and to support the selection of this 
issue as an appropriate topic for performance improvement. The narrative, in the additional 
information provided during and after the on-site review connected the reduction in emergency 
room visits for low-severity health care services to improved quality, accessibility, and timelines 
of care. The stated overarching goal of the project was to improve the quality of life of MC+ 


Members by reducing the excessive time spent in the emergency room. 
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A specific study population was not identified. However, the study was focused on all members 
that met the criteria. No group or individual in the MC+ population was excluded by the study. 
The narrative did incorporate two study questions to examine: 


= In Mercy MCts Medicaid population, will member education about available resources 
and the appropriate use of emergency room services: 
a. Decrease low severity visits; 
b. Decrease the emergency room rate/!000; and 
c. Increase primary care provider use. 


# Will discussions with Mercy MC+’s Medicaid providers about access and availability 
decrease the Emergency Room Visit Rates (ERVR)? 
Three specific and measurable indicators were identified in the project narrative. These 
included: 


= The proportion of Medicaid members accessing the emergency room for a low severity 
problem. 
= Emergency room visits per |,000. 
» The proportion of Medicaid providers with an emergency room visit ratio (ERVR) 
greater than 3.5(>60%). 
There was good presentation of the methods to be used to calculate measures and the 
benchmarks to be used for comparison. The narrative defined the numerators, denominators, 
and rate calculation. The number of months Medicaid members were enrolled in the MCO was 
factored into the definition of the denominator in the “emergency room visits per 1000” 


indicator. This did not serve to exclude members, but did assist in defining a quantifiable 


measure. 


It appears that the goals of the study are very specific. Long term outcomes are not defined. 
The original narrative introduces the concept of improved quality of life for members, but the 


identified indicators’ relevance to this goal is not explained. 


The narrative and calculation definition does not explain how the MCO will track specific 
members who are targeted for intervention. It is unclear if the gross data that will be used in 
the indicators outlined actually improved members quality of life, or if the MCO only achieves a 


reduction in frequency of emergency room visits. 
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No demographic information for participants of the study was identified. It was unclear whether 
only MC+ Members were part of the study. In the original information submitted, it appeared 
the participants would be included from other projects in other States. The data collection 
approach appeared to capture all enrollees to whom the study question applied. Participants 
were identified through claims and administrative databases available to the MCO. The actual 


sources of the data were not specified. 


The planned interventions were defined and included in the project narrative. These 
interventions included: developing a monthly report of members meeting the project criteria; a 
plan for telephone contacts with members that includes a call script to ensure that all members 
obtain consistent information; provider contacts to discuss the requirement for providing after- 
hour availability, reinforcing policy to not pay for emergency room transportation for non- 
emergent services; policy to make an exception to the 3-day call requirement for members 
needing PCP services; provider letters addressing ERVRs and services available; and home visits 
by a community partner to members failing other interventions. The interventions described 
are designed to have a positive impact on both member and provider behavior. The narrative 
did not present hypotheses to measure if the planned approach would have the desired affect. 
Barriers were not described and actions to overcome barriers were not addressed. The project 
narrative did not clarify how the effect of these interventions would be analyzed for any positive 


affect on member behavior or quality of life. 


Although detailed data management information and definitions were provided, no actual data 
analysis plan was presented. Baseline data from January through December 2003, and January 
through December 2004 were presented. It was possible to determine the source of some of 
the data, but this was not clearly stated on the form provided. The available information did not 
provide reliable or valid information for analysis. Planned data analyses should attempt to 
identify the significance of changes since the baseline was established. Since this project was just 


underway, and not complete there was no analyses of findings included. 


The study appeared to have moderate potential for producing credible findings. The QIA Form, 
even with the additional information provided after the on-site review, provided limited 
information about the entirety of the quality improvement and the PIP that was implemented. 


The information provided during interviews and after the on-site review indicates that this is an 
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active project with potential for identifying improvement. The format of the information 
provided does not satisfy all the requirements of the Protocol for Validating Performance 


Improvement Projects. 


The second PIP evaluated was “Customer Service — Member Service Call Center Quality.” This 
PIP was identified as a non-clinical intervention. The rational for selection of this project was 
based upon the results of the yearly Consumer Assessment of Health Plans Survey (CAHPS), 
which indicated that the rates of customer satisfaction for operations remained flat or declined. 
The MCO also performed some internal analysis of operations. This revealed problems that 
stemmed from the call center. For example, there was an increase in appeal decisions 
overturned due to inaccurate benefit information provided from the call center. The 
justification for selecting this topic for a performance improvement project was “To improve 
the quality of the call experience for members by assessing qualitative performance against call 
quality attributes. Individual and group performance monitoring provides call center managers 
with unbiased assessments of performance — a useful tool to reward progress.” The narrative 
did identify project goals as: 


= Improved verbal and written communication to the member; 

= Improved compliance with HIPAA privacy and security guidelines and preservation of 
member rights; and 

= Achieving excellence in areas of respect and dignity toward the member. 


There was no study question included in the information provided. The narrative did not make 
it clear how customer satisfaction was related to the proposed interventions at the call center, 
or how this was to be measured. The narrative did not delineate how this project would impact 


MC+ Members. No specific population was detailed in the information provided. 


The study narrative did not provide objective and clearly defined or measurable indicators. The 
one measure included was “Average Call Quality Rating.” It did not explain what this rating was 
or how it was achieved. It did state that the goal for performance improvement was to achieve 
an average call rating of >_ 98%. There was no explanation of what this number related to, or 


how this measure is achieved. 
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The study does not explain how it will measure any positive impact on members or member 
services. Sources of data are not revealed. The narrative alludes to a separate “QUIP” that will 
track progress toward call quality objectives.” It does not relate if the MCO will compare 
progress in the call center quality rates to the customer service rates from the annual CAHPS. 
All sources of data are not identified. The project does not provide an explanation of a 
systematic method of collecting valid and reliable data that represents the entire population to 
which the study applies. The documentation provided does include a survey instrument to be 
utilized for call monitoring. There is no accompanying explanation of how data collected from 


this instrument is analyzed. 


The interventions for staff monitoring, the development of training programs and initiatives, and 
the desired impact on member information and satisfaction were discussed during the on-site 
review. This information is not detailed in the project documentation. The information 
provided does not present a complete picture of the planned interventions and the expected 


outcomes. 


This is a twelve-month project that was not fully implemented until January 2006. There is no 


data available to be analyzed. No data analysis plan was provided in the narrative. 


This project is far from mature. It may have some potential for a positive impact on member 
services, but this is impossible to assess at the present time. During the on-site review the 
MCO was provided with comments about the difficulty of validating the PIP in the format 
presented. There is only a limited relationship between the intent, methods, and outcomes in 
the information provided. It is not possible to evaluate if this project will produce credible 


findings based on the information available. 
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STRENGTHS 


|. The first Performance Improvement Project, Emergency Room Utilization has the potential 
to positively impact services to members. 

2. The Emergency Room Utilization Project presented an excellent justification based on the 
national literature review and baseline data. 

3. The projects presented have the potential to lead to multi-year projects that will create 
growth in member services. 

4. The PIPs presented addressed both clinical and nonclinical issues. 


AREAS FOR IMPROVEMENT 


|. Improved development and provision of adequate narrative explaining and supporting the 
proposed or on-going activities presented would enhance the validity or potential of the 
projects presented. 

2. Incorporating all the data and measurements in a manner that related to the problem, 
interventions, outcomes, and conclusions would add credibility. 

3. There is a need to clearly state a study question in each project presented in order to focus 
the topic and guide the interventions developed. 

4. Clearly defined measure indicators must be included in all projects. 


RECOMMENDATIONS 


|. Utilize the “Conducting Performance Improvement Projects” protocol in reporting the PIP 
to ensure that all aspects of projects are covered in the narrative, or to explain any deficits. 

2. In development and reporting on non-clinical PIPs, ensure that the problem, the 
interventions, and intended outcomes truly impact member services, and are not just a 
method of improving MCO internal performance. 

3. Work on providing clear and reliable information and data to ensure that intended 
outcomes can be identified and supported. 
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7.2 Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Mercy MC+. Mercy MC+ submitted the requested documents on 
December 9, 2005. The EQRO reviewed documentation between December 9, 2005 and 
February 27, 2006. On-site review time was used to conduct follow-up questions and provide 


feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Baseline Assessment Tool (BAT) submitted by Mercy MC+ 

e Healthcare Research Associates’ (HRA) HEDIS Compliance Audit Report for 2005 

e Mercy MC+’s Information Systems (IS) policies and procedures pertaining to HEDIS 2005 rate 
calculation 

e Mercy MCts IS policies on disaster recovery 

e CRMS & HEDIS project plan 

e NCQA HEDIS software certification report 

e Medical record review process flowchart 

e CareEnhance Resource Management System (CRMS) data warehouse and Health Plan Reporter (HPR) 
Manual 


The following are the data files submitted by Mercy MC+ for review by the EQRO: 


e ADV _Filel.txt 
e ADV _File2..txt 
e CIS Filel.txt 
e CIS File2.txt 
e CIS File3.txt 
e WIS _Filel txt 
e WI5 File2.txt 
e WI5 File3.txt 


Interviews 

On Wednesday, March Ist, 2006, the EQRO conducted a site visit and interviews at Mercy 
MC+ with Patricia Snodgrass, Charles McLaughlin, Dave Reisinger, and Donna Osdyke. This 
group was responsible for calculating the HEDIS 2005 performance measures. The objective of 
the visit was to verify the methods and processes behind the calculation of the three HEDIS 
performance measures. The information systems (IS) management policies and procedures for 
rate calculation were consistently evaluated with the CMS Final Protocol for the Validating 


Performance Measures. This included both manual and automatic processes of information 
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collection, storing, analyzing and reporting. Due to the requirements of their data warehouse, 


Mercy MC+ was unable to show the EQRO the HEDIS 2005 data during the site visit. 


FINDINGS 


Mercy MC+ calculated the Childhood Immunization Status, Combination #2 and the Well-Child 
Visits measures using the Hybrid Method. MCO to MCO comparisons of the rates of 
Childhood Immunization Status Combination #2, Well-Child Visits, and Annual Dental Visit 
measures were conducted using two-tailed z-tests. For comparisons that were statistically 
significant at the 95% confidence interval (Cl), the z-score (z), the upper and lower confidence 


intervals (Cl), and the significance levels (p < .05) are reported. 


The rate for the HEDIS 2005 Childhood Immunization Status, Combination #2 reported to the 
SMA and the State Public Health Agency (SPHA) was 38.93%. This figure was comparable to the 
statewide rate for all MC+ MCOs (28.17%; z = -.52; statewide 95% Cl: 27.83%, 50.03%; n.s.). 


The rate for Mercy MC+ for the HEDIS 2005 Well-Child Visits in the First 15 months of Life 
measure was 43.80%, comparable to the statewide rate for all MC+ MCOs (38.42%; z = .04, 
statewide 95% Cl: 31.27%, 56.33%; n.s.). The 2005 HEDIS rate for Mercy MC+ for the Annual 
Dental Visit was 20.52%, significantly lower than the statewide rate for MC+ MCOs (29.76%, z = 
-|.62; statewide 95% Cl: 16.24%, 24.80%; p < .001). 


The following sections summarize the findings of the process for validating each of the 
performance measures in accordance with the Validating Performance Measures Protocol. The 
findings from all review activities are presented according to the EQRO validation activity. The 
findings for each measure discussed within the activities are deemed appropriate. Please refer 


to the tables in the main report for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

For all three measures, Mercy MC+ was found to meet all criteria for producing complete and 
accurate data (see Attachment V: Data Integration and Control Findings). There were no 
biases or errors found in the manner in which Mercy MC+ transferred data into the repository 


used for calculating the HEDIS 2005 measures. 
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Documentation of Data and Processes 

Mercy MC+ used NCQA-certified software from McKesson Inc. for the sampling and calculation 
of HEDIS measures. The EQRO was provided with a demonstration of the Health Plan 
Reporter (HPR) (the application module for rate calculation), as well as the CareEnhance 
Resource Management System (CRMS) data warehouse. The information contained in the data 
warehouse and shown to the EQRO during the site visit was the current HEDIS data, and not 
the data being validated by the EQRO. Mercy MC+ stated that they were unable to show the 
EQRO the data they were validating because a system upgrade required the MCO to purge all 
the prior year’s data. Data and processes used for the calculation of measures were adequate 
(see Attachment VII: Data and Processes Used to Calculate and Report Performance 
Measures). Mercy MC+ met nearly all criteria applicable for the three measures. There were 
two criteria which were not met. The first involved the use of statistical significance testing to 
document changes in performance over time. The second criteria was to utilize this information 
to support claims of improvement or stability in performance over time. Although many 
auditors conduct statistical tests to examine various aspects of data validity, such tests should be 
used by the Mercy MC+ to assess the significance of change related to quality improvement 
activities and operational changes. The use of NCQA-certified calculation software is 


considered adequate for calculating the rates. 


Processes Used to Produce Denominators 

Mercy MC+ met all criteria for the processes employed to produce the denominators of all 
three performance measures (see Attachment X: Denominator Validation Findings). This 
involved the selection of members eligible for the services being measured. The denominator 
file for the Childhood Immunization Status measure contained a total of |,912 members; Mercy 
MC+ reported a total of |,309 eligible members on the DST. 411 records were selected for the 
sample; Mercy MC+ only provided the EQRO with service codes and service dates for the 41 | 
members, therefore it is difficult to determine why there was a discrepancy between the eligible 
members found by the EQRO and those identified by the MCO. There were no members 
represented more than once and the dates of birth and enrollment were valid. The 
denominator file for the Well Child Visits measure contained a total of 1,739 members; Mercy 
MC+ reported a total of |,272 eligible members on the DST. Again, 411 records were selected 


for the sample; Mercy MC+ only provided the EQRO with service codes and service dates for 
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the 41 1 members, therefore it is difficult to determine why there was a discrepancy between 


the eligible members found by the EQRO and those identified by the MCO. 


For the Annual Dental Visit measure, there were a total of 18,704 eligible members reported by 
Mercy MC+; the EQRO validated a total of 15,488 eligible members. There were no duplicate 


members included and the dates of birth and enrollment were in the valid ranges. 


Processes Used to Produce Numerators 

Data ranges for the qualifying events (e.g., immunizations, well-care visits, and asthma 
medications) as specified by the HEDIS 2005 criteria were reviewed in the files submitted by 
Mercy MC+ (see Attachment XIll: Numerator Validation Findings). Medical record reviews 
were conducted for the Childhood Immunization Status and Well-Child Visit measures. 

For the HEDIS 2005 Childhood Immunization Status Combination #2 measure, Mercy MCt+ 
appropriately included administrative events from the State Public Health Immunization Registry 
(MOHSAIC). A total of 1,912 unique members were validated in the file; Mercy MC+ reported 
a total of 1,309 eligible members on the DST. The MCO only provided numerator data for the 
4|1 members sampled; of those, all 90 of the reported administrative hits were validated. The 
dates of birth and dates of service were within the valid ranges. Of the 70 numerator hits 
reported by medical records, the EQRO sampled 30 for validation. All 30 of the medical 
records requested were received for review, 29 of which were validated by the EQRO. 
Therefore, 96.67% of the medical records could be validated for a total of 68 hybrid hits. The 
rate calculated by the EQRO based on validated administrative and hybrid hits was 38.36%, an 
overestimate of .57%. However, the accuracy of this bias is in question, as Mercy MC+ only 
requested 182 of the 321 records in the sample that did not contain administrative hits. Of the 
182 records they requested, they only received 148. It is difficult to estimate how many 
additional hits may have been found through the hybrid method if the records had been 
requested for review. In effect, the MCO excluded 139 records from the onset of the medical 


record review process. 


For the HEDIS 2005 Well-Child Visits measure, a total of |,272 eligible members were reported 
on Mercy MC+’s DST. The EQRO found I,739 eligible members in the files provided by Mercy 
MC+. The MCO only provided numerator data for the members selected for the sample of 41 | 
members. Of those, all 133 of the reported administrative hits were validated. The dates of 


birth and dates of service were within the valid ranges. Of the 47 numerator hits reported by 
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medical records, the EQRO sampled 30 for validation. All 30 records were received, 21 of 
which were validated. Of the 9 records that did not have the required six well-child visits to be 
considered a hit, four (4) records contained 5 visits, two (2) records contained 4 visits, one (I) 
record contained 3 visits, and two (2) records contained 2 visits. With 70.00% of the medical 
records validated, a total of 33 hybrid hits were found by the EQRO. The rate calculated by the 
EQRO based on validated administrative and medical record hits was 40.36%. This resulted in 
an overestimated bias of 3.44% by Mercy MC+. Again, the accuracy of this bias is in question, as 
Mercy MC+ only requested |75 of the 236 records in the sample that did not contain 
administrative hits. Of the |75 records they requested, they only received 97. It is difficult to 
estimate how many additional hits may have been found through the hybrid method if the 
records had been requested for review. In effect, the MCO excluded 6! records from the 


onset of the medical record review process. 


The HEDIS 2005 Annual Dental Visit was the third measure validated. There were a total of 
3,838 unique numerator hits reported, of which 3,833 were validated. The dates of birth, 
enrollment and services were within the valid ranges. The final rate calculated by the EQRO 


was 20.49%, an overestimated bias of 0.03%. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Childhood Immunization Status and the Adolescent Well- 
Child Visits measures. Attachment XIl; Impact of Medical Record Review Findings and 
Attachment XV: Sampling Validation Findings were completed for each of these measures. 
There were no replacements, contraindications, or exclusions reported by the MCO. All 


criteria for sampling were met for the two measures. 


Submission of Measures to the State 

Mercy MC+ submitted the DST for each of the three measures validated to the SPHA, (Missouri 
Department of Health and Senior Services; DHSS) in accordance with the Code of State 
Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA 


Quality Improvement Strategy. 
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Determination of Validation Findings and Calculation of Bias 

Table 57 shows a summary of the estimated bias in the rates calculated by Mercy MC+. All 
three measures were overestimated by Mercy MC+, however, the Annual Dental Visit measure 
was only slightly overestimated, and therefore, is considered fully compliant. The overestimates 
of both the Childhood Immunization Status, Combination, #2 and Well-Child Visits measures 


were within the 95% confidence interval for the rates reported by the MCO. 


Table 57 - Estimate of Bias in Reporting of HEDIS 2005 Measures 


Measure Estimate of Bias Direction of Estimate | 
Childhood Immunization Status, Combination #2 Overestimate 


Well-Child Visits in first 15 months of Life Overestimate 
Annual Dental Visit 





Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all 
data sources. These sources were summarized in the Final Performance Measure Validation 
Worksheet for each measure (See Table 58). The overestimates found in calculation of the 
Childhood Immunization Status, Combination #2 and Well-Child Visits in the First 15 Months of 
Life measures were within the confidence interval, and therefore, were determined to be Not 


Valid. The Annual Dental Measure was Fully Compliant and valid. 


Table 58 - Final Audit Rating for HEDIS 2005 Performance Measures 


Measure Final Audit Ratinc 
Childhood Immunization Status, Combination #2 Substantially Compliant 


Well-Child Visits in first 15 months of Life Substantially Compliant 

Annual Dental Visit Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by the MCO. Not Valid = Measure deviated from State specifications such 
that the reported rate was significantly biased. This designation is also assigned to measures for which no rate was 
reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Mercy MC+ used NCQA-certified HEDIS software, CRMS/HPR of McKesson Inc. Most of 
the data for HEDIS measures are stored in the CRMS warehouse from which HPR is used 
to automatically calculate rates. There were clearly defined data fields within the 
application, as it automatically feeds the rates required by the State into the HEDIS Data 
Submission Tool. 


2. Numerous validation checks in the reporting process, both manual and automatic were 
conducted within the software system. There were sufficient edit checks for data entry, 
which helped in minimizing errors. 
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3. There were well-documented policies and procedures for the HEDIS rate calculation 
measures. 


4. There were full data integration, retrieval and analysis processes in place. This facilitated 
efficient collection of data from claims, credentialing provider, pharmacy, lab, utilization, 
ophthalmic and dental data sources. The State Public Health Immunization Registry 
(MOHSAIC) was used for collection of immunization data, and is well-integrated into the 
system. 


5. There was a focused software development and technical analysis group for data 
collection, programming and analysis. 


6. Data analysis incorporated tests of statistical significance to assess whether the observed 
changes in rates were related to a specific intervention. 


AREAS FOR IMPROVEMENT 


|. The MCO should have a better understanding of the mechanisms and output results of 
the CRMS/HPR system. Although the software is NCQA-certified, the MCO should 
display an interest in assuring that the system’s outputs are accurate. 


2. The MCO should provide the EQRO with the methodology and processes used to 
calculate HEDIS rates; this information is provided by other plans who utilize the same 
software vendor. 


3. The HEDIS 2005 Annual Dental Visit measure rate was significantly lower than the 
average for all MC+ MCOs. 


RECOMMENDATIONS 


|. Improve the collection of medical records from providers by stipulating or reinforcing 
policies and procedures referring to medical records storing, archiving and retrieval. 


2. Responsible managers should improve ownership and control of the CRMS/HPR system 
to have a better understanding of query structures. There is a need for the Mercy MCt+ 
to have a better understanding of the CRMS/HPR system and how this relates to the 
HEDIS calculation rates. 


3. The MCO should maintain a development machine or second system with the past HEDIS 
year’s data for validation until the EQRO concludes the on-site audit. 


4. The MCO should provide the EQRO with the methodology utilized by McKesson to 
calculate measures, or provide a McKesson representative to answer questions during the 
site visit that may be “proprietary”. 
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7.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical Fields? 


For the Medical Claim type, there were 105,046 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


|. The Outpatient Claim Type field was 100.00% complete, accurate, and valid. 
The Outpatient Recipient ID field was 100.00% complete, accurate, and valid. 


The Outpatient First Date of Service field was |00.00% complete and accurate, and 99.32% valid. 
There were 710 invalid dates of service ranging from 4/29/2005-- 12/3 1/2005. 


The Outpatient Last Date of Service field was |00.00% complete and accurate, 99.29% valid. 
There were 571 invalid dates of service ranging from 4/01/2005--9/2 1/2005. 


The Outpatient Units of Service field was |00.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 100% complete and accurate, and 99.85% valid. 
Invalid procedure codes consisted of 154 “99601”, three “99602”, and one “K0635” entries. 


The Outpatient Place of Service field was 100.00% complete, accurate, and valid. 


The first Diagnosis Code field was 99.99% complete, accurate, and valid. Nine fields were left 
blank. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 44.53% complete, accurate, and valid. 
The remaining fields were blank (incomplete, inaccurate, and invalid). The third, fourth, and 
fifth Diagnosis Code fields were 0.00% complete, accurate, and valid. The remaining fields 
were blank (incomplete, inaccurate, and invalid). 


For the Dental claim type, there were | 1,592 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate, 


and valid. 
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For the Home Health claim type, there were 29 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. All fields examined were |00.00% complete, 


accurate, and valid except the following. 


The Outpatient Claim Type field was 100.00% complete, accurate, and valid. 

The Outpatient Recipient ID field was 100.00% complete, accurate, and valid. 

The Outpatient First Date of Service field was |00.00% complete and accurate, and valid. 
The Outpatient Last Date of Service field was 100.00% complete, accurate and valid. 
The Outpatient Units of Service field was |00.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 0.00% complete, accurate and valid. All fields were 
blank (incomplete, inaccurate, and invalid). 


The first Diagnosis Code field was 100.00% complete, accurate, and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 89.66% complete, accurate, and valid. 
The remaining fields were blank (incomplete, inaccurate, and invalid). The third Diagnosis 
Code field was 82.76% complete, accurate, and valid. The remaining fields were blank 
(incomplete, inaccurate, and invalid). The fourth and fifth Diagnosis Code fields were 0.00% 
complete, accurate, and valid. The remaining fields were blank (incomplete, inaccurate, and 
invalid). 


For the Inpatient claim type, there were 5,929 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 


The Inpatient Claim Type field was 100.00% complete, accurate, and valid. 
The Recipient ID field was 100.00% complete, accurate, and valid. 
The Admission Type field was 100.00% complete, accurate, and valid 


The Admission Date field was 100.00% complete and accurate, and 95.99% valid. There were 
238 invalid dates ranging from 12/15/2005 --12/31/2005. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 98.23% (with 104 entries of 
“99999999”. Valid values were present 96.21% of the time. In addition to the invalid 
“99999999” entries, 120 invalid dates ranged from 04/01/2005-- 04/05/2005 


The Bill Type field was 100.00% complete, accurate, and valid. 
The Patient Status field was 100.00% complete, accurate, and valid. 
The first Diagnosis Code field was 100.00% complete, accurate, and valid. 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (75.54%, 55.20%, 0.00%, and 
0.00%, respectively). 
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The First Date of Service field was 100.00% complete and accurate, and 95.99% valid. There 
were 238 invalid dates ranging from | 2/15/2005 --12/31/2005. 


The Last Date of Service field was 100.00% complete and accurate, and 97.00% valid. There 
were | 78 invalid dates of service ranging from 04/01/2005--04/30/2005. 


The Revenue Code field was 99.95% complete, accurate, and valid, with 3 blank fields 
(incomplete, inaccurate, and invalid). 


The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were 69,901 encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. All fields examined were 100.00% 
complete, accurate, and valid with the exception of the Revenue and second through fifth 
Diagnosis Code fields. The Revenue Code field was 99.97% complete, accurate, and valid, with 
23 blank fields (incomplete, inaccurate, and invalid). The second, third, fourth, and fifth Diagnosis 
Code fields fell well below the 100% threshold for completeness, accuracy, and validity 
established by the SMA (52.88%, 24.73%, 0.00%, and 0.00%, respectively). The remaining fields 


were blank (incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were 65,892 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. All fields examined were |00.00% complete, accurate, and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Mercy MC+, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. There 
were very few errors encountered in the critical fields examined across all claim types. For the 
Medical claim type, the Outpatient First Date of Service and Outpatient Last Date of Service 
fields had invalid entries (see above findings) and there were nine blank Diagnosis code fields in 
the first Diagnosis Code field. The Inpatient claim type had invalid Admission and Discharge 


Date fields and three blank Revenue Code fields in the Inpatient claim type. 
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To what Extent do the Claims in the State Encounter Claims Database 
reflect the Information Documented in the Medical Record? 

What is the Fault/Match Rate between State Encounter Claims and Medical 
Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 
types for the period of January |, 2005 through March 31, 2005 for medical record review. Of 
the 105,046 Medical encounter claim types in the SMA extract file for January |, 2005 through 
March 31, 2005, 100 encounters were randomly selected. Providers were requested to submit 
medical records for review. There were 76 medical records (76.0%) submitted for review. 
Encounters for which no documentation was submitted were unable to be validated. The match 
rate for procedures was 43.0%, with a fault rate of 57.0%. The match rate for diagnoses was 


76.0%, with a fault rate of 24.0%. 


What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review for procedure and diagnosis 
codes was conducted. For the diagnosis code in the medical record, the reasons for diagnosis 
codes not matching the SMA extract file were incorrect information (n = 24). The diagnosis 


code listed did not match the descriptive information in the record. 


For the procedure code in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = 23) and incorrect codes (n = 20). Examples of missing information included no 
code, codes listed that were not supported, or codes that did not match the procedure 


description. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, Medical and Dental 
claim types for Mercy MC+ were consistent with the average for all MC+ MCOs. The 
Outpatient encounter claim type rate was significantly higher than the average for all MC+ 
MCOs, while the rates for Home Health, Inpatient, and Pharmacy claim types were significantly 


lower than the average for all MC+ MCOs. This could indicate a possible access to care for 
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preventive services that is related to a higher need for inpatient services; or may be associated 


with claims administration. 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing Mercy MC+s (MHP) encounter 
data to the SMA encounter claim extract file because there was not an identical encounter 
number to key the two files. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. While MHP did submit the data in the requested format 
(see Appendix 6) for the MC+ Managed Care Members represented in the encounter claim 
sample selected by the EQRO for validation, however no internal control number (ICN) was 


included. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While the MC+ MCO did submit the data in the requested format, there are a number of ways 
to improve the data quality by improving the database system. One variable that is not 
currently represented is that of a unique line number. To match up specific lines of data (each 
service provided within an encounter), this requires a unique number for each service provided 


for each member. 
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STRENGTHS 


The data was submitted in the specified format which allowed BHC to assess all claim 
types associated with Mercy MC+. 


The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


The critical fields evaluated for the Dental claim type was 100.00% complete, accurate, and 
valid. 


AREAS FOR IMPROVEMENT 


The Outpatient First Date of Service and Last Date of Service fields in the SMA encounter 
claims extract file for the Medical claim type were 99.319% and 99.29% valid, respectively. 


For Outpatient Home Health claims, procedure codes were missing for all 29 encounters. 


For the Inpatient claim type, the Discharge Date field and Revenue Code field were 
96.21%, and 99.95 valid, respectively. The Revenue Code contained three blank fields. 


For Outpatient Hospital claims, the Revenue Code field was 99.97% complete, accurate, 
and valid. Twenty-three fields were blank. 


Mercy MC+ demonstrated a significantly lower rate than the average of all MC+ MCOs 
for validation of the SMA encounter claims extract file against the medical record for 
diagnosis codes and procedure codes. Reasons for medical records not matching were 
primarily related to missing data. 


RECOMMENDATIONS 


Examine and revise as needed internal system edits for invalid procedure codes in the 
NSF/CMS 1500 file layout and run validity checks after the programming of new edits, 
and ensure completion of Units of Service field. 


Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 


For the Inpatient claim type (UB-92 file layout), improve the rate of valid Discharge 
Dates to flag invalid entries of “99999999” and blank entries for the Units of Service 
field. Error checks for the Diagnosis Code field should also be conducted to ensure no 
blank fields. 


It is recommended that Mercy MC+ examine the possible reasons for the lower rates of 
outpatient services (Medical and Outpatient Hospital claim types) and the higher rates of 
inpatient services through a non-clinical performance improvement project aimed at 
analysis of encounter claim and utilization data as well as access to care indicators. 
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7.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract documents with the staff of the Division of Medical Services. On-site review 
time was used to conduct interviews with those who oversee the daily practices of the MCO to 
ensure that documentation is developed and practices occur within the scope of the contract 


and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The Division of Medical Services supplied: 
» State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

= Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection of actions filed in the first quarter of 2005. 

#» 2004 Annual Quality Improvement Program Evaluation 


Additional documentation Mercy MC+ included: 
» MC+ Marketing Policy 
=» MC+ Marketing Plan/2005 and 2006 Community Outreach Plan 
= Contract Policy Submittal Log/Mercy DMS Policy Tracking Log 
= Care Issues Report 
» Narrative on methods of performance for MHP programs 
=» Member Newsletters 
# NCQA Activity: Improving the Quality of Life of Members Living With Asthma 
= Executive Summary HEDIS Missouri Regions 
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» 2004 Corporate Quality Improvement Program 
# Policies: Disenrollment Procedures, Mercy MC+ Pharmacy Lock-In, Oversight of 
Delegated Entities, Emergency and Post Stabilization Care -MC+ 


Unity Managed Mental Health supplied the following documents regarding their services to 
Mercy MC+ members: 


» Unity Managed Mental Health — 2005 Operations Reporting (for Mercy MC+s MC+) 
= Access Standards and Compliance Policy 

#» 2005 Member Satisfaction MHP MC+ Report 

# 2005 Readmission Report 

# 2005 Provider Record Survey Outcomes 

# 2005 Provide Access Survey 

# 2005 Update -MC+ Pregnant Women Care Coordination 


Interviews 


Interviews were conducted with the following group: 


Plan Administration 
Steve Mead, Product Manager, Medicaid 
Dr. Debbie Zimmerman, Chief Medical Officer 
Pat Snodgrass, Director, Performance Improvement 
Liz Scott, Business Analyst 
Anna Dmuchovsky, V.P., Health Resources 
Susan Meiner, Director, Benefit Coordination 
Donna Hauler, Manager, Member Services 
Debbie Todd, Member Services 
Cindy Johnson, Director, Care Coordination 
Sam Fenner, MC+ Administrator 


Mental Health 
Scott Frederick, PHD, Director, Managed Mental Health - UMMH 
Marge Viehland, Manager, Utilization Management and Q.|. - UMMH 
Cindy Johnson, Care Coordination, Mercy MC+ 
Liz Scott, Business Analyst, Mercy MC+ 


FINDINGS 

Enrollee Rights and Protections 

Mercy MC+ reported that they purchased software to assist in tracking all policies and 
procedures, including those that must be reviewed on an annual basis. This will ensure timely 
submission to the SMA for their approval. The MCO reported that they also added a 
compliance officer to the staff to assist in this process. These changes are not reflected in the 
information obtained about Mercy MC+’s 2005 activities. 
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Mercy MC+ was working with a new disease management program that enabled a one-on-one 
relationship with nurses. This program enables complex care coordination and self-education 
focusing on lifestyle and teaching. Enhanced software was also utilized to capture any member 
with elevated lead levels. Mercy MC+ was attempting to identify children with elevated blood 
lead levels during any healthcare provider encounter. The MCO experienced some difficulty 
incorporating contacts from the St. Louis County Health Department, which they continued to 
refine. Mercy MC+ was actively engaged in improving their relationship with the St. Louis City 
Department of Health in an attempt to strengthen the provision of elevated blood lead level 
interventions through that source. The MCO also made efforts to improve awareness, and 
provide education about the need for blood lead level testing with school nurses and counselors 


in Jefferson County. 


The MCO was making new efforts to identify members and obtain valid telephone numbers and 
addresses for them. Mercy MC+ contracted with a group, KAMA, Inc., an organization that 
updated and augmented their files once each quarter. This company accessed utility company 
records and other sources to obtain the most current contact information available. The MCO 
reported that this process assisted them in locating both new and current members more 


effectively. 


Mercy MC+ initiated a program linking the Compliance Department with network pharmacies. 
This assisted in the identification of members who may have a problem with prescribed 
medications. Case management and pharmacy lock-in occurred to assist the member when this 
problem was identified. A process was put in place during 2005 for notification of providers 


when a member was identified as having a problem with drug abuse or polypharmancy issues. 


The rating for Enrollee Rights and Responsibilities (30.8%) reflects policy that was not complete 
or not submitted to the SMA for their review and approval. Mercy MC+ provided a policy 
tracking log and draft policies to be submitted to the SMA for review and approval. Although 
some improvement in the submission was noted, compliance with the MC+ Medicaid Managed 


Care contract requirements and the federal regulations was not complete. 
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Table 59 - Subpart C: Enrollee Rights and Protections Yearly Comparison 





: Mercy MC+ 

Federal Regulation 
438.100(a) Enrollee Rights: General Rule 1 1 
438.10(b) Enrollee Rights: Information Requirements 1 1 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 
438.10(c)(4,5) Language and Format: Interpreter Services 1 1 
438.10(d)(1)(i) Information Requirements: Format/Easily 1 1 
Understood 
438.10(d)(1)(ji)and (2) Information Requirements: Format Visually 1 5 
Impaired, and Limited Reading Proficiency 
438.10(f) Information for All Enrollees: Free Choice, etc. 1 1 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 1 1 
Plans 
438.10(i) Special Rules: Liability for Payment/Cost Sharing 1 1 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee Communications 2 2 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 1 1 
438.100(b)(3) Right to Services 0 1 
438.100(d) Compliance with Other Federal/State Laws 1 2 
Number Met 2 4 
Number Partially Met 10 9 
Number Not Met 1 0 
Rate Met 15.4% 30.8% 


Note: 0 = Not Met; I= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Behavioral Health 

The Behavioral Health Organization (BHO) providing mental health services to Mercy MC+’s 
members was Unity Managed Mental Health (UMMH). During 2005, the two organizations 
collaborated on a number of initiatives. Case management services were targeted to members 
who were pregnant. These members were identified through risk assessments, welcome calls, 
calls coming into Member Services, and through the quality review process. Mercy MC+ 
worked to improve its relationship with providers to encourage their assistance in completing 
the risk assessment tool. Identified members then went into a collaborative case management 


program with UMMH. 


For all dually served members, Mercy MC+ and UMMH created a triage process to identify 
members with both physical and mental health care needs. To facilitate this process, early in 


2005, Mercy MC+ and UMMH implemented a shared case management program. The two 
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organizations shared the same health information system, thus making this arrangement possible. 
The BHO and MCO staff met and developed a methodology for role sharing focused on 
approaches to members who needed both types of services. A screening process was 
developed that placed members into one of four categories. These included members who: 
refused case management; were accepted into an outreach class; received an outreach call and 
follow-up; or received a call and appropriate literature. As a result of improved information 
sharing, the Pregnant Women Care Coordination Program experienced an acute increase in 
members and nearly doubled in size. UMMH ensured that all pregnant women identified were 
referred to Mercy MC+ for health case management. Mercy MC+ referred any woman 


identified with mental health or substance abuse problems to UMMH for follow-up services. 


Quality Assessment and Performance Improvement 


Access Standards 

Mercy MC+ continued to struggle to find certain specialists to join their network. This included 
pediatric neurosurgery, adolescent psychiatrists, orthopedic surgeons, and pain management. 
Mercy MC+ ensured that members had access to these specialists when appropriate, by 
negotiating rates and utilizing out-of-network physicians. The MCO noted that Barnes-Jewish- 
Christian Medical Centers were added to their network during 2005, which provided access to 
Washington University Medical School. Mercy MC+ was questioned about adding nurse 
midwives to their network. They were not able to recruit this service, and indicated they did 
not view this aspect of their network as a priority. The MCO stated that they have no prior 
authorization requirement for child psychiatry. Ifa member contacted the MCO requesting this 
service, a referral was made to UMMH to ensure that a provider was located in a timely 


manner. 


Case management services remained an important focus for Mercy MC+. Case management 
was provided to all members with catastrophic illness, those in need of targeted disease 
management, and members with special healthcare needs. The MCO had a program for 
physician to physician outreach for babies at risk through a link with neonatologists ensuring 
better discharge planning and aftercare. This service was coordinated by discharge planners 
who were housed in various hospitals. Mercy MC+ Member Services and Case Management 
departments worked to recognize unique member needs and respond in a timely and efficient 


manner. Examples included recruitment and placement of a Vietnamese speaking home health 
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nurse and locating a dentist for a child with dental carries upon referral from an emergency 


room physician. 


Mercy MC+ worked with area universities to develop exercise and after-school programs for all 
ages. They developed a fitness magazine called “Pulse” that focused on children’s health issues. 
A website was developed that offered a “Kids Health, Teen Health” area and information for 
parents. The MCO was contacting area school superintendents and principles offering these 


programs. 


The ratings for Access Standards (52.9%) continued to reflect policies that were not complete, 
or that have not yet been reviewed and approved by the SMA. Mercy MC+ did provide several 
policies that they submitted to the SMA prior to the on-site review. Although these updates 
were noted, the policies had not been reviewed or approved by the SMA. The MCO had not 


achieved compliance with the MC+ Medicaid Managed Care contract or federal regulations. 
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Table 60 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 


Mercy MC+ 


Federal Regulation 





438.206(b)(1)(i-v) Availability of Services: Provider Network 1 1 
438.206 (b) (2) Access to Well Woman Care: Direct Access 0 1 
438.206(b)(3) Second Opinions 2 2 
438.206(b)(4) Out of Network Services: Adequate and Timely 5 5 
Coverage 

438.206(b)(5) Out of Network Services: Cost Sharing 1 1 
438.206(c)(1)(i-vi) Timely Access 1 1 
438.206(c)(2) Provider Services: Cultural Competency 2 2 
438.208(b) Care Coordination: Primary Care 2 2 
438.208(c)(1) Care Coordination: |Identification 2 2 
438.208(c)(2) Care Coordination: Assessment 1 1 
438.208(c)(3) Care Coordination: Treatment Plans 1 1 
438.208(c)(4) Care Coordination: Direct Access to Specialists 1 1 
438.210(b) Authorization of Services 1 2 
438.210(c) Notice of Adverse Action 1 2 
438.210(d) Timeframes for Decisions, Expedited Authorizations 1 2 
438.210(e) Compensation of Utilization Management Activities 2 2 
438.114 Emergency and Post-Stabilization Services 0 1 
Number Met 6 9 
Number Partially Met 9 8 
Number Not Met 2 0 


Rate Met 35.3% 52.9% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Structures and Operation Standards 
Mercy MC+ maintained a thorough credentialing process. All credentialing and reports of 
delegated credentialing were completed within required timeframes. All policy and practices 


regarding disenrollment were submitted to the SMA and approved. 


At the time of the 2004 review, Mercy MC+ indicated that they intended to begin a quality 
improvement initiative that would involve a Medicaid review committee that included members. 
The MCO developed an MC+ Medicaid Managed Care work group internally, but has not yet 


involved members as participants. The MCO used this group to provide feedback and to 
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problem-solve about barrier reductions in member services. Mercy MC+ continued 
development of quality improvement efforts to increase the rate of EPSDT examinations. The 
MCO used exception reporting to remind providers of examinations that were due or overdue. 
The MCO reported that they increased their EPSDT numbers, but recognized that some 
examinations occurred at local health departments or other providers and were never reported 


as such. 


Mercy MC+ continued to utilize the Blue Ribbon Physician’s Network. Appointment standards 
and quality of care were monitored and the provider group’s per member per month rate was 
adjusted based on their productivity. University Physicians and Barnes-Jewish-Christian Medical 


Group were added to this group in 2005. 


The ratings for Structure and Operation Standards (90%) reflected substantial improvement in 
submitting all required policy to the SMA for their review and approval. Mercy MC+ continued 
their solid practice in this area. With completion of all required policy, the MCO will be fully 
compliant in meeting the requirements of the MC+ Medicaid Managed Care contract 


requirements and federal regulations. 


Table 61 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


Mercy MC+ 


Federal Regulation 





438.214(a,b) Provider Selection: Credentialing/Recredentialing 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 
438.214(d) Provider Selection: Excluded Providers 


438.214(e) Provider Selection: State Requirements 


438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 
limitations 


438.56(c) Disenrollment Requested by the Enrollee 
438.56(d) Disenrollment: Procedures 

438.56(e) Disenrollment: Timeframes 

438.228 Grievance System 

438.230(a,b) Subcontractual Relationships and Delegation 


Number Met 
Number Partially Met 
Number Not Met 
Rate Met 30.0% 90.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Measurement and Improvement 

Mercy MC+ continued to utilize nationally accepted practice guidelines. The MCO’s Internal 
Quality Improvement Committee continued to review medical records to ensure that these 
guidelines were correctly implemented. The MCO completed all required policy and practice in 


the area and was considered compliant in the area of practice guidelines. 


The MCO had an integral quality assessment and improvement program within the organization. 
They utilized information from all subgroups within Mercy MC+ to analyze barriers that existed 
for providers and members. The Performance Improvement Department then engaged in 
problem solving to ensure that: providers were held to the highest standards of performance; 
that the MCO system supported the provision of sound healthcare services; and that members’ 


quality, timeliness and access to these services met all requirements. 


Mercy MC+ did submit two PIPs for validation. Additional information about these studies was 
obtained at the time of the on-site review. In the case of the non-clinical PIP submitted, there 
was some question about the validity and completeness of the study. These concerns are 


discussed in the appropriate section of this report. 


The MCO submitted all required information to complete the Validation of Performance 
Measures, as requested. Mercy MC+ continued to operate a health information system within 
the guidelines of that protocol. All encounter data requested was provided in a format that 
could be validated. The details of each of these areas of validation can be reviewed within 


specific sections of this report. 


Ratings for the Measurement and Improvement section (100%) reflect that all required policy 
and practice existed to meet the requirements of the MC+ Medicaid Managed Care contract 


and the federal regulations. 
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Table 62 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison 


Mercy MC+ 


Federal Regulation 





438.236(b)(1-4) Practice Guidelines: Adoption 2 2 
438.236(c) Practice Guidelines: Dissemination 2 2 
438.236(d) Practice Guidelines: Application p 2 
438.240(a)(1) QAPI: General Rules 2 2 
438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 5 5 
Improvement and PIPs 

438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 1 2 
438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 2 2 
438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare 5 5 
Needs 

438.240(e) QAPI: Program Review by State NA NA 
438.242(a) Health Information Systems 2 2 
438.242(b)(1,2) Health Information Systems: Basic Elements 1 

438.242(b)(3) Health Information Systems: Basic Elements 1 2 
Number Met 8 10 
Number Partially Met 3 0 
Number Not Met 0 0 


Rate Met 72.1% 100.0% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; I= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Grievance Systems 

Four member grievances and six provider complaints were reviewed at the time of the on-site 
review. One member did not receive their identification card, which was reordered and sent to 
the member. In another case a member could not locate an orthopedic surgeon. The MCO 
located an out-of-network provider who accepted the Medicaid reimbursement for providing 
the necessary services. One member was unhappy with their original primary care physician 
(PCP), stating this provider was rude and unprofessional. The situation was resolved with a 
referral to a new PCP. The last member presented to the PCP with back and ovary pain. The 
provider’s staff would not schedule an appointment and stated the member needed to see a 
specialist. The specialist contacted by the member would not see this individual without a PCP 


referral. The member complained that the PCP staff treated her rudely and was not 


“ohare Performance Management Solutions Group 


Solutions Group 
iT 


it eee A division of Behavioral Health Concepts, Inc. 253 


Missouri MC+ Managed Care External Quality Review Section 7 
Report of Findings — 2005 Mercy MC+ 


sympathetic to her frustration. In this member’s file Mercy MC+ wrote that the staff were 
“Advised and the complaint would be noted.” The member was informed by the MCO that if 
she wanted another PCP, they would provide a listing, but advised that “they were not 


responsible for providers’ staffing.’ 


The documentation in these member records was clear and dated in sequence of contacts and 
correspondence. However, no follow-up letters to members were located. With the 

exception of the final grievance discussed, the responses generally appeared to be appropriate 
and timely. In the final grievance there was no additional information in the member’s file and 


no apparent follow-up with the member or original provider. 


Of the six provider complaints reviewed, three were initially denied, but upon inspection found 
to be emergent in nature and were paid. Another denial for services was overturned when the 
member's enrollment date was corrected. The next complaint was overturned due to late filing, 
but it was overturned. The final complaint concerned a claim that was upheld due to no prior 
authorization. Documentation within the provider records was more difficult to follow and no 


correspondence was located in these files. 


The MCO indicated that policy was developed for provider complaints, grievances and appeals. 
However, the appearance of the records reviewed indicated that this policy was not followed at 


the beginning of 2005. 


The rating for the Grievance Section (89%) reflects the lack of appropriate information properly 
filed in the records for members, or for provider complaints. Required policy is in place and has 
been submitted to the SMA for final approval. The MCO will be fully compliant with this section 


when the records of this section are in the order as required by their own policy. 
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Table 63 - Subpart F: Grievance Systems Yearly Comparison 
Mercy Health Plan 


Federal Regulation 





438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2 
438.404(a) Grievance System: Notice of Action - Language and 5 5 
Format 
438.404(b) Notice of Action: Content 2 2 
438.404(c) Notice of Action: Timing 2 2 
438.406(a) Handling of Grievances and Appeals: General 5 5 
Requirements 
438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 
438.408(a) Resolution and Notification: Basic Rule 
438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 
438.408(d)(e) Resolution and Notification: Grievance and Appeals - 

2 2 
Format and Content of Notice 
438.408(f) Resolution and Notification: Grievances and Appeals - 5 5 
Requirements for State Fair Hearings 
438.410 Expedited Resolution of Appeals 2 2 
438.414 Information about the Grievance System to Providers and 5 1 
Subcontractors 
438.416 Recordkeeping and Reporting Requirements 2 2 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 18 16 
Number Partially Met 0 2 
Number Not Met 0 0 
Rate Met 100% 88.9% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 


Summary / Follow-up 

Mercy MC+ improved in completing and submitting policy to the SMA for review and approval, 
however, the MCO continued to have policies and procedures to complete and submit. It was 
noted that no sense of urgency about completing these requirements to achieve compliance 
with either the MC+ Medicaid Managed Care Contract or federal regulations existed. Mercy 
MC+ ratings for Grievance Systems declined as the result of insufficient maintenance and 


practice identified in the records reviewed. 
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Mercy MC+ exhibits a sense of professionalism and compassion in their work with their 
members. Positive aspects of the practice observed at the MCO are overshadowed by the lack 


of attention to completing required policy and procedure. 


STRENGTHS 


|. Improved submission of required policies and procedures in an effort to comply with state 
contractual requirements and federal regulations. 

2. Continued partnership with Unity Managed Mental Health, including the current 
collaboration in the area of case management. 

3. Case Management practices, particularly the approach MHP took in the areas of special 
projects, lead, asthma, and all special healthcare needs. 

4. Efforts to control misuse of prescription drugs including utilization of pharmacy data, 

identification of members, and contacting providers. 

All MC+ community outreach efforts. 

6. Focusing resources on Kids Health and Teen Health, including the utilization of web based 
information. 


wa 


AREAS FOR IMPROVEMENT 


|. Timely and complete submission of required policy and procedures. 

2. Need for network development for specialty physicians and PCPs. The Provider listing in 
the Member Handbook indicated that there was one county bordering St. Louis County 
with only one PCP having an open panel. The MCO indicated that they were below the 
requirement of 85% of available PCPs. 

3. Include any correspondence to members in the grievance and appeal files. Ensure that 
members are treated with respect when they have an involved problem. Provide 
supervisory oversight and reflection in the file of follow-up with those involved when a 
problem exists. 

4. Increase documentation of how provider complaints, grievances and appeals are handled. 
Include correspondence in the file. 


RECOMMENDATIONS 


|. The MCO reported new software and methods for tracking annual and required policy 
submission deadlines. Utilize this tool to continue improvement in this area. 

2. Obtain approval for, and implement, developed policy regarding provider complaints, 
grievances, and appeals. 

3. Continue efforts to maintain current demographic data on members, including their current 
addresses and telephone numbers. Use of a contractor, and other methods, to focus on 
this issue, indicated the MCO’s commitment to provide member services. 

4. Continue community based partnerships and provider education efforts to improve the 
incidence of EPSDT examinations and implementing preventive health measures with 
children/members who may not access Mercy MC+ services. 
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8.1 Performance Improvement Projects 


The previous sections of the 2005 EQRO report present the purpose and objectives, technical 
methods, procedures for evaluation, and MCO to MCO comparisons of analyses, findings, and 
recommendations for each of the protocols based on the aggregate analysis. This section of the 
EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed 
Care Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


METHODS 
Document Review 


HealthCare USA supplied the following documentation for review: 


= Performance Improvement Project 2005: 2005 Lead Performance Improvement Project 

= Performance Improvement Project 2005: 2004 Pre-Authorization Performance 
Improvement Project 

# “Important Information” Lead Information Brochures 

# Provider Information Letter, Poster 

» “NewsLEADer” for Members, and all lead-related handouts and marketing materials 

"= HealthCare USA Provider Satisfaction Survey 2003 

" Authorization Directory 

= Coventry QuickGuide “Back Office Operations” 


Interviews 

Interviews were conducted with the project leaders for each Performance Improvement Project 
(PIP) by the EQRO team on March I, 2006 during the on-site review, and included the 
following: 


Jackie Inglis — VP Health Services 

Cathy Krueger — Supervisor, Quality Improvement 
Cindy Butler — Manager, Health Services 

April Gross — Supervisor, Complex Case Management 
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The interviewees shared information on the validation methods, study design, and findings. 
Technical assistance regarding study design and presentation of findings was provided by the 
EQRO. The following questions were addressed: 


=» Who was the Project Leader? 

= How was the topic identified? 

= How was the study question determined? 

=» What were the findings? 

=» What was the intervention? 

= What was the time period of the study? 

=» Was the intervention effective? 

» What does HCUSA want to study or learn from their PIPs? 
The maturity of the PIPs submitted for validation was not complete at the time of the initial 
request for this information in October 2005. The MCO was instructed that they could submit 
additional information that included outcomes of the intervention. Additional clarifying written 


information was received from the MCO after the on-site review. 


FINDINGS 

The first PIP evaluated was the “2005 Lead Performance Improvement Project.”” The study 
topic, increasing the rate of blood lead level testing for children at both |2 and 24 months of 
age, was explained in detail. The background and explanation was presented, including the 
MCO’s literature and data reviews, as well as references, supporting the selection of the topic. 
The study topic justification examines the depth of the problem of blood lead poisoning, and the 
inadequate attention to appropriate testing that often accompanies this issue. National and 
statewide statistics were considered in making the decision for choosing the topic and approach 


for this study. 


The MC+ Medicaid Managed Care population turning ages |2 or 24 months of age during 2005 
was the target of this study. The rational for targeting children of this age was presented, 
including a reference to the Centers for Medicare and Medicaid Services (CMS) and Missouri 
State requirements for blood lead level testing at these ages. The study question was: “Does 
member and provider education regarding lead testing at |2 and 24 months have an impact on 
the overall rate of lead testing at 12 and 24 months?” The MCO anticipated that blood lead 
level testing rates would increase for both age groups with the implementation of the 
educational initiative. Two separate interventions are implied with the study question 
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presented. The narrative does not differentiate how the impact or effectiveness of either 


intervention will be measured to specify individual influence. 


Indicators and goals are presented in a table included in the presentation. There is no narrative 
explanation, which leaves some questions about the meaning of the indicators. No sampling 
techniques were used. The study population is described as all “children eligible with 
HealthCare USA who turn |2 and 24 months old 2005.” The study will also target primary care 


physicians, pediatricians, and family practice physicians for education. 


The data to be collected includes claims data pulled from the MC+ MCO data warehouse for 
eligible members obtaining blood lead chemistry tests within three months of their 12 month or 
24 month birthday. The numerators and denominators are defined. The study provides no 
additional details of the data sources or how a systematic method for ensuring valid and reliable 
data will be determined. The study does not provide a prospective data analysis plan, although 
some actual data analysis is included. The narrative does identify barriers to the improvement 
strategy, and a proposed intervention for member, providers, the community, and MC+ MCO 


staff. 


Data analysis was performed for figures received for the 2005 calendar year, and are compiled 
by MC+ Medicaid Managed Care region, as well as in an aggregate manner. Claims were 
analyzed quarterly. This study was planned to continue during calendar year 2006 and is not 
considered complete. The actual annual data for the baseline year 2004 and the first year of the 
intervention 2005, were included. Percentages of change were also indicated. A 2 x 2 
contingency table (chi-square statistical analysis) was used to determine statistical significance 


between the 2004 baseline and the 2005 re-measurement data. 


It is possible that the positive change identified was the result of the interventions presented, 
but a strong case for this was not presented. Improvements in member behavior was difficult to 
identify, and it could not be determined if this was the result of member or provider changes. 
The study does discuss a number of environmental factors that can impact the results of this 
study. It acknowledges the barriers to success, particularly in the Eastern MC+ Managed Care 
Region. The narrative described other community based interventions that occurred 


independently of MCO efforts, but the impact of these differences were not factored into the 
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results presented. The information presented does not clarify how any of the issues identified 
as barriers or the other factors that impact compliance in the Eastern Missouri MC+ Region 
might be addressed. Changes as the result of identifying these issues might be included as new 
components to address in the next implementation cycle of the study, but this was not 
addressed. The information presented, including the data analysis, provides moderate 
confidence that this project will have a positive impact on the topic. The variables and 
environmental factors listed as barriers, and the use of multiple and differing interventions, 


creates a number of questions about the true cause for measured change. 


The second PIP evaluated was titled “2004 Pre-Authorization Performance Improvement 
Project.” This study was considered non-clinical and focused on improving call center and pre- 
authorization efficiency linked to improved continuity of care for MC+ Medicaid Managed Care 
members. The project narrative clearly identified how provider satisfaction is tied to access and 
availability of services for members. This was identified as a multi-year project, with baseline 
statistics from 2004 with the goal of improving, stabilizing and maintaining pre-authorization 
telephone statistics. Background research or any literature review was not included in the 
information presented. The rationale presented for justifying the decision to choose this topic 
was based on internal data and performance review. The providers who were the focus of this 
study included primary care providers, OB/GYNs and orthopedic surgeons. Although this was a 
non-clinical PIP, not directly impacting members, the array of physicians targeted in the 


proposed study had the potential to improve services for all enrolled populations. 


The study question included was “Will operational adjustment improve telephone statistics and 
provider satisfaction with pre-authorization?” The question does not relate to the access or 
availability of services to members, leaving this relationship implied. The study did present 
indicators and goals for achieving increased provider satisfaction in the timeliness for answering 
calls, and in the overall pre-authorization system. It did not look at direct indicators, such as 
provider complaints, grievance, and appeal system, which could identify problems. Overturned 
denials may have been another area of data that could be used in measuring outcomes for this 
topic, but it was not included in how the MC+ MCO is defining measurable improvement. The 
study defined the population as all providers requesting any pre-authorization services. No 
actual sampling occurred. However, the types of providers receiving surveys included only 


PCPs, Obstetricians, and Orthopedic specialists. The study claims that this group of physicians 
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“represents the majority of the HCUSA network of providers who provide the majority of 
services to HCUSA members.” No data or other supporting documentation was provided to 


support this claim. 


The data collection methodology was included and centered on gathering statistics from the 
MC+ MCOs automated call distribution system, and a Provider Satisfaction Survey. The Survey 
was included in the documentation provided. Data was entered into an Access database to 
track responses and increases or decreases over time. Chi-Square Statistical Analysis was used 
to determine significance from year to year. Timeframes for survey distribution were not 
specified. It can be assumed that these would occur annually, and in all three MC+ Regions. 
The study design detailed all parts of the analysis process and included a prospective data 
analysis plan. Interventions and improvement strategies were documented in the narrative. 


Barriers were described and reasonable interventions to address these issues were included. 


An analysis of the findings was included for the 2003 baseline information and the 2004 re- 
measurement data. There were no results from the 2005 year included in the original 
information provided. At the time of the on-site review additional information from the 2005 
call center monitoring was presented. All information presented was well documented, labeled 


and explained. 


The plan was for this study to be completed at the end of 2005, although all final figures were 
not available. All available results indicated that there were no declining trends and that 
evidence of sustained improvement would be available when all data was completed. The MC+ 
MCO was including all activities involved as interventions as part of their routine call center 
operations. This project used a research approach to solve a non-clinical problem. The project 
had the long term goal of improving member services through improving services to providers. 
The MC+ MCO achieved statistically significant improvement on both telephone statistics and 
provider satisfaction. The changes that occurred as part of the interventions became part of the 
daily operations of the pre-authorization department. This non-clinical PIP was successful and 


indicated sustained improvement in MC+ MCO operations. 
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STRENGTHS 


Valid topics were chosen for performance improvement for the MCO, and were supported 
in the narrative including the literature review presented for the first PIP reviewed. 


The documentation provided prior to the site visit and additional information supplied at the 
time of the on-site review provided detailed information that enhanced the validation 
process. 


The Performance Improvement Projects truly attempted to enhance the quality of services 
provided by HCUSA to its members. 


Data analysis provided sound evidence that “real” improvement occurred throughout the 
projects. 


AREAS OF IMPROVEMENT 


Although follow-up information included a thorough narrative, the original information was 
more difficult to follow and evaluate. 


The focus of the lead initiative was St. Louis City. The MCO serves three regions, and 
admittedly three distinct populations. There were some interventions in other regions, but 
St. Louis was clearly targeted. It is important to be inclusive in choosing and implementing 
initiatives so all members benefit. 


Ensure that the prospective data analysis plan is included in the project planning 
documentation. 


RECOMMENDATIONS 


Continue with plans to use HEDIS data definitions for continuous measurement and at least 
quarterly re-measurement and interpretation of effectiveness of the intervention. 


Continue with the practice of analyzing PIP effectiveness, ensuring that the interventions for 
improvement have been incorporated into regular HCUSA practice so real improvement 
continues. 
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8.2 Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validating 
Performance Measures Protocol for HealthCare USA. HealthCare USA submitted the 
requested documents on December 9, 2005, after a conference call between the EQRO and 
HealthCare USA additional information was supplied to the EQRO on January 9, 2006 to 
replace the data files submitted on December 9, 2005. The EQRO reviewed documentation 
between December 9, 2005 and February 27, 2006. On-site review time was used to conduct 
follow-up questions and provide feedback and recommendations regarding the performance 


measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The HealthCare USA Baseline Assessment Tool (BAT) for the HEDIS 2005 data reporting year 

e HealthcareData.com LLC’s Compliance Audit Report for HEDIS 2005 

e HealthCare USA’s information systems policies and procedures with regard to calculation of HEDIS 
2005 rates 

e HealthCare USA meeting minutes on information system (IS) policies 

e Asample of Catalyst’s production logs and run controls 

e National Council on Quality Assurance (NCQA)-certified HEDIS software certification report from 
Catalyst Technologies 

e Data field definitions & claims file requirements of the Coventry Corporate Data Warehouse 

e Data files from the Coventry Corporate Data Warehouse containing the eligible population, 
numerators and denominators for each of the three measures. 

e HEDIS 2005 Data Submission Tool 

e HEDIS 2005 product work plan 


The following are the data files submitted by HealthCare USA for review by the EQRO: 


e adv denominator — hcusa central.txt 
e adv denominator — hcusa eastern.txt 
e adv denominator — hcusa western.txt 
e adv events — hcusa central.txt 

e adv events — hcusa easternl.txt 

e adv events — hcusa western.txt 

e wl5 denominator — hcusa central.txt 
e wl5 denominator — hcusa eastern.txt 
e wl5 denominator — hcusa western.txt 
e wl5 events — hcusa central.txt 

e wl5 events — hcusa eastern.txt 

e wl5 events — hcusa western.txt 

e cis denominator — hcusa central.txt 

e cis denominator — hcusa eastern.txt 
e cis denominator — hcusa western.txt 
e cis events — hcusa central.txt 

e cis events — hcusa eastern.txt 

e cis events — hcusa western.txt 

e Copy of field long names.xls 
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Interviews 

The EQRO conducted on-site interviews at HealthCare USA in St. Louis Thursday, March 2, 
2005 with Cathie Krueger, Lisa Baird, and Laura Frasier. Also available by phone were Rina 
David-Claytor and Geoff Welsh, who represented the software vendor Catalyst Technologies. 
This group was responsible for calculating the HEDIS 2005 performance measures. The 
objective of the visit was to verify the methods and processes behind the calculation of the 


three HEDIS 2005 performance measures. 


FINDINGS 

HealthCare USA calculated each of the three measures validated by using the Administrative 
Method and by region of operation (Eastern, Central, and Western). HealthCare USA failed to 
provide the detailed enrollment information for all three measures, including enrollment dates 
and any gaps in enrollment. This information is necessary to validate continuous enrollment and 
determine the eligible population for each measure. HealthCare USA did provide the EQRO 
with a denominator file that contained a list of eligible members for each measure. Athough 
these files were processed through HealthCare USA’s Catalyst software and should contain only 
eligible members for each measure (as Catalyst is a NCQA-certified software), the EQRO was 
not able to independently validate the denominator files because enrollment history/dates were 
not supplied. Additionally, HealthCare USA did not provide service codes that could be 
validated against the HEDIS Technical Specifications. The service codes that were provided had 
been processed through the Catalyst software and were no longer in the ICD-9 or CPT Code 
format. The information provided did not include the CPT Codes or ICD-9-CM Codes, but a 
category or description of what services those service codes represented. It is necessary for 
the EQRO to receive service codes in the CPT or ICD-9-CM format not only because that is 
what was request, but also because those formats are the industry accepted standard and the 
only way for the EQRO to ensure that what they are matching between plans is the same 


information. 


As noted, HealthCare USA calculated each of the three measures validated using the 
Administrative Method and by region of operation (Eastern, Central, and Western). Results 
were reported for each of the three regions and in the aggregate for the entire MCO, as the 
EQRO is charged with providing MCO level comparisons. MCO to MCO comparisons of the 


rates of Childhood Immunization Status Combination #2, Well-Child Visits, and Annual Dental 
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Visit measures were conducted using two-tailed z-tests. For comparisons that were statistically 
significant at the 95% confidence interval (Cl), the z-score (z), the upper and lower confidence 


intervals (Cl), and the significance levels (p < .05) are reported. 


HealthCare USA Aggregate: 
The aggregate rate was calculated by the EQRO from the information reported by HealthCare 


USA to the SMA and the State Public Health Agency (SPHA). The HEDIS 2005 Childhood 
Immunization Status, Combination #2 measure was 18.46%, significantly lower than the 
statewide rate for all MC+ MCOs (28.17%; z = -1.88; 95% Cl: 7.36%, 29.56%; p < .0Ol). The 
agsregate rate for Health Care USA for the HEDIS 2005 Well-Child Visit measure was 41.50%, 
comparable to the statewide rate for all MC+ MCOs (38.42%; z = -.10; 95% Cl: 28.97%, 58.43%; 
n.s.). The rate for Health Care USA for the HEDIS 2005 Annual Dental Visit measure was 
28.89%, comparable to the statewide rate for all MC+ MCOs (29.76%; z = -.17; 95% Cl: 24.61%, 
33.17%; n.s.). 


HealthCare USA Central Missouri Region: 
The rate reported by HealthCare USA to the SMA and the State Public Health Agency (SPHA), 


for the HEDIS 2005 Childhood Immunization Status, Combination #2 measure in the Central 
Missouri region was 34.91%, comparable to the statewide rate for all MC+ MCOs (28.17%; z = - 
19; 95% Cl: 28.81%, 46.01%; n.s.). The rate for Health Care USA in the Central Missouri region 
for the HEDIS 2005 Well-Child Visit measure was 59.85%, significantly higher than the statewide 
rate for all MC+ MCOs (38.42%; z = .99; 95% Cl: 47.32%, 72.38%; p > .95). The rate for Health 
Care USA in the Central Missouri region for the HEDIS 2005 Annual Dental Visit measure was 
22.41%, significantly lower than with the statewide rate for all MC+ MCOs (29.76%; z = -1.29; 
95% Cl: 18.13%, 26.69%; p < .O1). 


HealthCare USA Eastern Missouri Region: 
The rate reported by HealthCare USA to the SMA and the State Public Health Agency (SPHA), 


for the HEDIS 2005 Childhood Immunization Status, Combination #2 measure in the Eastern 
Missouri region was 14.96%, significantly lower than the statewide rate for all MC+ MCOs 
(28.17%; z = -2.12; 95% Cl: 3.86%, 26.06%; p < .O|). The rate for Health Care USA in the 
Eastern Missouri region for the HEDIS 2005 Well-Child Visit measure was 38.68%, comparable 
to the statewide rate for all MC+ MCOs (38.42%; z = -.27; 95% Cl: 26.15%, 51.21%; n.s.). The 
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rate for Health Care USA in the Eastern Missouri region for the HEDIS 2005 Annual Dental 
Visit measure was 30.30%, comparable to the statewide rate for all MC+ MCOs (29.76%; z = 
07; 95% Cl: 26.02%, 34.58%; n.s.). 


HealthCare USA Western Missouri Region: 
The rate reported by HealthCare USA to the SMA and the State Public Health Agency (SPHA), 


for the HEDIS 2005 Childhood Immunization Status, Combination #2 measure in the Western 
Missouri region was 22.01%, significantly lower than the statewide rate for all MC+ MCOs 
(28.17%; z = -1.65; 95% Cl: 10.91%, 33.11%; p < .Ol). The rate for Health Care USA in the 
Western Missouri region for the HEDIS 2005 Well-Child Visit measure was 27.06%, significantly 
lower than the statewide rate for all MC+ MCOs (38.42%; z = -.95; 95% Cl: 14.53%, 39.59%; p < 
Ol). The rate for Health Care USA in the Western Missouri region for the HEDIS 2005 Annual 
Dental Visit measure was 21.77%, significantly lower than the statewide rate for all MC+ MCOs 


(29.76%; z = -1.41; 95% Cl: 17.49%, 26.05%; p < .O1). 


The EQRO did validate each of the performance measures in accordance with the Validating 
Performance Measures Protocol. The findings from all review activities are presented according 
to the EQRO validation activity, with the findings for each measure discussed within the 
activities as appropriate. Please refer to the tables in the aggregate report for activities, ratings, 


and comments related to the Attachments. 


Data Integration and Control 

The information systems management policies and procedures for rate calculation were 
evaluated consistent with the Validating Performance Measures Protocol. This included both 
manual and automatic processes of information collection, storing, analyzing and reporting. For 
all three measures, HealthCare USA was found to meet nearly all the criteria for producing 
complete and accurate data (see Attachment V: Data Integration and Control Findings). The 
one criterion that was not met involved “detailed computer queries, programming logic, or 
source code used to identify the population or sample for the denominator and/or numerator.” 
There were no biases or errors found in the manner in which HealthCare USA transferred data 
into the repository used for calculating the HEDIS 2005 measures. HealthCare USA used an 
NCQA-certified software vendor, Catalyst, for the HEDIS 2005 measure calculation process. 
The EQRO was provided with a demonstration of the Quality Spectrum Hybrid Reporter™, the 
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application module for rate calculation, and with Coventry’s corporate data warehouse. The 
use of NCQA-certified reporter software, which has been certified through a process of test 
files, indicates that the program specifications, codes, and measure parameters are adequate for 
validly reporting the rates. However, the codes, program specifications, and measure 
parameters could not be independently validated by the EQRO as we were unable to view the 
data prior to it being processed by the software. HealthCare USA supplied the EQRO with 
information that had already been processed once by their data system. The information 
provided did not include the CPT Codes or ICD-9-CM Codes, but a category or description of 
what services those service codes represent. It is necessary for the EQRO to receive service 
codes in the CPT or ICD-9-CM format not only because that is what was request, but also 
because those formats are the industry accepted standard and the only way for the EQRO to 


ensure that what they are matching between reported rates is the same information. 


Documentation of Data and Processes 
It is assumed that the data and processes used for the calculation of measures were adequate 
(see Attachment VII: Data and Processes Used to Calculate and Report Performance 


Measures). HealthCare USA met all criteria that applied for all three measures. 


Processes Used to Produce Denominators 

The EQRO was unable to determine whether HealthCare USA met the criteria for the 
processes employed to produce the denominators of all three performance measures (see 
Attachment X: Denominator Validation Findings), because this involves the selection of eligible 
members for the services being measured. Although the denominators for all three measures in 
the final data files supplied to the EQRO were consistent with those reported on the DST, the 
EQRO could not validate the denominator due to the absence of member enrollment dates. 
Member enrollment dates are necessary to determine the eligible population for HEDIS 
measures. HealthCare USA assured the EQRO that the software they used to process the data 
utilized the HEDIS 2005 Technical Specifications. This was not verifiable by the EQRO due to 
HealthCare USA's failure to provide the enrollment data and service codes requested. The 


eligibility numbers below were based on age ranges only. 
In the Central Missouri region, |,166 eligible members were reported and found for the 
Childhood Immunization Status, Combination #2 measure. A total of 1,051 eligible members 
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were reported and validated by the EQRO for the Well Child Visit measure. There were 
13,786 eligible members reported and found by the EQRO for the denominator of the Annual 


Dental measure in the Central Missouri region. 


In the Eastern Missouri region, 5,783 eligible members were reported and found for the 
Childhood Immunization Status, Combination #2 measure. A total of 5,083 eligible members 
were reported and validated by the MCO for the Well-Child Visits measure. There were 
78,555 eligible members reported and found for the denominator of the Annual Dental 


measure. 


In the Western Missouri region, 309 eligible members were reported and found by the EQRO 
for the Childhood Immunization Status, Combination #2 measure. A total of 340 eligible 
members were reported and validated by the EQRO for the Well-Child Visits measure. There 
were 2,968 eligible members reported and found for the denominator of the Annual Dental 


measure for the Western Missouri region. 


Across all three regions, 7,258 eligible members were reported and found for the Childhood 
Immunization Status, Combination #2 measure. A total of 6,474 eligible members were 
reported and 6,435 were found for the Well-Child Visits measure. 95,309 eligible members 
were reported and found for the denominator of the Annual Dental measure across the regions. 
Age ranges, dates of birth, and medical events were validated solely for those members who 
met HEDIS 2005 criteria. Enrollment dates and continuous enrollment were not independently 


validated by the EQRO, but it is believed these were programmed correctly. 


Processes Used to Produce Numerators 

All three measures were calculated using the Administrative Method. The EQRO is unable to 
determine whether the measures included the appropriate data ranges for the qualifying events 
(e.g., immunizations, well-care visits, and dental visits) as specified by the HEDIS 2005 Technical 
Specifications (see Attachment XIII: Numerator Validation Findings). No medical record 


reviews were conducted or validated. 
HealthCare USA appropriately included administrative events from the State Public Health 
Immunization Registry (MOHSAIC) in calculating the Childhood Immunization Status, 
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Combination #2 measure. For this measure, a total of |,340 administrative hits were reported 
and 1,105 were found. This resulted in a rate of 15.22%, with an overestimate of 3.247%. 
HealthCare USA did not supply CPT codes, ICD-9-CM codes, and/or HCPCS/CDT-3 codes to 
the EQRO in order to facilitate validation. The service codes were requested by the EQRO in 
the original Performance Measures Data Submission Request (See Appendix 3). The service 
codes are an industry standard that is used by the EQRO to match the service the MC+ MCO 
delivered to its member(s) with the HEDIS Technical Specifications “Codes to Identify 
Childhood Immunizations”. Without the acceptable service codes, the EQRO was unable to 
determine if valid services had been rendered to HealthCare USA’s MC+ members. The dates 


of birth range were valid. 


For the HEDIS 2005 Well-Child Visits measure, there were a total of 2,687 administrative hits 
reported and 2,528 hits found. This resulted in a rate of 39.05%, with an overestimate of 2.46%. 
HealthCare USA did not supply CPT codes, ICD-9-CM codes, and/or HCPCS/CDT-3 codes to 
the EQRO in order to facilitate validation. The service codes were requested by the EQRO in 
the original Performance Measures Data Submission Request (See Appendix 3). The service 
codes are an industry standard that is used by the EQRO to match the service the MC+ MCO 
delivered to it’s member(s) with the HEDIS Technical Specifications “Codes to Identify Well- 
Child Visits”. Without the acceptable service codes, the EQRO was unable to determine if valid 
services had been rendered to HealthCare USA’s MC+ members. The dates of birth range were 


valid. 


The number of administrative hits reported for the HEDIS 2005 Annual Dental Visit measure 
was 27,536; the EQRO found 27,679. This resulted in a rate of 29.04%, with an underestimate 
of -0.15%. The codes for numerator events were not provided and could not be validated. The 
service codes were requested by the EQRO in the original Performance Measures Data 
Submission Request (See Appendix 3). The service codes are an industry standard that is used 
by the EQRO to match the service the MC+ MCO delivered to its member(s) with the HEDIS 
Technical Specifications “Codes to Identify Dental Visits”. The dates of birth were valid and in 


the correct range. 
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Sampling Procedures for Hybrid Methods 
No medical record reviews were conducted or validated. Attachment XIl; Impact of Medical 
Record Review Findings and Attachment XV: Sampling Validation Findings do not apply to the 


Administrative Method. 


Submission of Measures to the State 

HealthCare USA submitted the DST for each of the three measures to the SPHA (the Missouri 
Department of Health and Senior Services) in accordance with the Code of State Regulations 
(19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the SMA Quality 


Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

As previously noted, the MCO underestimated the Annual Dental Visit measure and 
overestimated both the Childhood Immunization Status and Well-Child Visits measures. 
However, the EQRO was unable to completely validate any of these measures due to the 


MCO’s non-compliance with the production of information requested by the EQRO. 


Table 64 - Estimate of Bias in Reporting of HEDIS 2005 Measures 


Measure Estimate of Bias 
Childhood Immunization Status, Combination #2 Overestimate 





Well-Child Visits in first 15 months of Life Overestimate 
Annual Dental Visit none 


Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all 
data sources summarized in the Final Performance Measure Validation Worksheet for each 
measure. All three measures calculated were Not Valid as they did not comply with State 


specifications, due to the MC+ MCO not providing the required service codes. 


Table 65 - Final Audit Rating for HEDIS 2005 Performance Measures 


Measure SEI W-leliangst-talate 
Childhood Immunization Status, Combination #2 Not Valid 


Well-Child Visits in first 15 months of Life Not Valid 

Annual Dental Visit Not Valid 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by the MCO. Not Valid = Measure deviated from State specifications such 
that the reported rate was significantly biased. This designation is also assigned to measures for which no rate was 
reported; Not Applicable = No MC+ Members qualified for the measure. 
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STRENGTHS 


|. There were significantly higher rates of HEDIS 2005 Well-Child Visit rates in the Central 
Missouri region, compared to the average for all MC+ MCOs. This rate was higher than the 
National Medicaid rates for this measure. 


2. HealthCare USA employed NCQA-certified HEDIS reporter software, Quality Spectrum ™ 
of Catalyst Technologies. Most of the data for HEDIS measures are stored in the Coventry 
IDX data warehouse from which Quality Spectrum is used to automatically calculate rates. 


3. The Quality Spectrum™ is a very well-designed application based in a SAS environment. On 
the whole, the application was very transparent and user friendly. The codes are updated as 
they are released by Catalyst. 


4. The Catalyst application had the ability to provide detailed and in-depth information for 
“drilling down” and evaluation purposes. Even though the codes running behind the 
program were proprietary, the structure of the repository was very informative and 
provided the interviewers with information on flow charts for calculations; event, rule, and 
metadata definitions. 


5. There were clearly defined data fields within the application, which automatically feeds the 
rates as required by the State in the HEDIS Data Submission Tool. 


6. There were numerous quality checks in the reporting process done manually with a 
separate abstract of the data files. This helps in minimizing errors. 


7. Efficient data integration, retrieval and analysis processes were in place. Members were 
identified with a unique primary key that avoids duplication errors. Datasets from 
CareMark, the pharmacy vendor; Quest Labs, the laboratory vendor and the dental vendors 
were used and integrated to Quality Spectrum for calculation of final rates. 


8. There was a good disaster recovery plan in place. 


9. MOHSAIC, the State Public Immunization Registry database, was used for collection of data 
regarding immunizations. The files were received as a text file and well integrated into the 
system. MOHSAIC data is loaded in a very specific format into the data warehouse. 
MOHSAIC is estimated to account for approximately 14% of the Adolescent Immunization 
Status rate. 


10. HealthCare USA had well-documented procedures for the HEDIS 2005 rate calculation 
measures. 


| 1. Upon review of preliminary validation findings for the performance measure validation, 
HealthCare USA recognized the importance of using the Hybrid Method to calculate the 
Adolescent Immunization Status measure and plans to use this method in the future. 


|2. HCUSA followed the EQRO’s 2004 recommendation and Integrated HEDIS rate 
documentation procedures into its corporate IS policies. 
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AREAS FOR IMPROVEMENT 


|. HealthCare USA should provide data to the EQRO in the format in which it was requested. 
By not doing so, the EQRO was unable to completely validate the Performance Measures. 


2. HealthCare USA used the administrative method for the calculation of all HEDIS 2005 
measures. Medical record review is likely to provide unique information regarding 
adolescent immunizations that is not captured in the State Public Immunization Registry for 
events prior to the implementation of the MC+ Managed Care Program and claims systems. 


3. The HEDIS 2005 Childhood Immunization Status, Combination #2 rate was significantly 
lower in both the Eastern and Western Missouri regions than the average for all MC+ 
MCOs. 


4. The HEDIS 2005 Well-Child Visits in the First 15 Months of Life measure rate was 
significantly lower in the Western Missouri region than the average for all MC+ MCOs. 


5. The HEDIS 2005 Annual Dental Visit measure rate was significantly lower in the Central 
Missouri region than the average for all MC+ MCOs. 


RECOMMENDATIONS 


|. The use of the Hybrid Method for the calculation of Childhood Immunization Status is 
strongly recommended. HealthCare USA should explore the possible reasons for low rates 
of administrative hits for the Childhood Immunization Status measure. It is the EQRO’s 
understanding that for the HEDIS 2006 season HealthCare USA will be utilizing the hybrid 
method for childhood and adolescent immunizations. 


2. Provide data to the EQRO in the format requested; data cannot be truly validated according 
to the CMS Protocols if it is accepted in a format that has already been processed by a 
software program. 


& Performance Management Solutions Group 
eee A division of Behavioral Health Concepts, Inc. 275 


Missouri MC+ Managed Care External Quality Review Section 8 
Report of Findings — 2005 HealthCare USA 


8.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical Fields? 


For the Medical claim type, there were 426,225 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 


The Outpatient First Date of Service field was |00.00% complete and accurate, and 99.90% valid. 
There were 426 invalid dates of service ranging from 01/01/2005 — 12/31/2005. 


The Outpatient Last Date of Service field was |00.00% complete and accurate, and 99.90% valid. 
There were 432 invalid dates of service ranging from 04/01/2005 — 10/03/2005. 


The Outpatient Units of Service field was |00.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 100.00% complete and accurate, and 99.72% valid. 
The following are the |,204 invalid entries found. 





Frequency Code 
903 99600 
198 99601 

5 B4034 

27 B4035 

4 B4036 

66 H1000 

| K0635 
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The Outpatient Place of Service field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% (with rounding) complete, accurate and valid. Three 
fields were blank. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA 
for this validation. The second Diagnosis Code field was |2.74% complete, accurate and 
valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The third 
Diagnosis Code field was 5.15% complete, accurate and valid (incomplete, inaccurate, and 
invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 2.06% complete, accurate and valid. The remaining fields were 
blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code field was 0.00% 
complete, accurate and valid. 


For the Dental claim type, there were 60,406 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate 


and valid. 


For the Home Health claim type, there were 64 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. All fields examined except the fourth and fifth 
Diagnosis Code fields were 100.00% complete, accurate and valid. The fourth and fifth 
Diagnosis Code fields fell well below the 100% threshold for completeness, accuracy, and 


validity established by the SMA (79.69% for each). 


For the Inpatient claim type, there were 53,367 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


The Inpatient Claim Type field was |100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 
The Admission Type field was 00.00% complete, accurate and valid. 


The Admission Date field was 100.00% complete and accurate, and 93.60% valid. There were 
3,415 invalid dates ranging from 06/17/2005 — 12/31/2005. 


The Discharge Date field was 100.00% complete with the correct number of characters (size). 
The correct type of information (date format) was present 97.09% (with I,551 entries of 
99999999"). Valid values were present 93.50% of the time. In addition to the invalid 
“99999999” entries, the 1,918 invalid dates ranged from 04/01/2005 — 04/27/2005. 


The Bill Type field was 100.00% complete, accurate and valid. 


The Patient Status field was 100.00% complete and accurate, and 99.93% valid. There were | 
invalid value of “00”, 7 invalid values of “61” and 28 invalid values of “63”. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 
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The second, third, fourth, and fifth Diagnosis Code fields fell below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (99.73%, 98.74%, 89.20%, 
72.41%, respectively). 


The First Date of Service field was 100.00% complete and accurate, and 94./0% valid. There 
were 2,829 invalid dates of service ranging from 10/29/2005 — 12/31/2005. 


The Last Date of Service field was 100.00% complete and accurate, and 94.80% valid. There 
were 2,/75 invalid dates of service ranging from 04/01/2005 — 04/28/2005. 


The Revenue Code field was 99.88% complete, accurate, and valid. There were 63 invalid blank 
fields. 


The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were 240,549 encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. 


The Outpatient Claim Type field was |00.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete, accurate and valid. 
The Last Date of Service field was 100.00% complete, accurate and valid. 
The Units of Service field was 100.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 100.00% complete, accurate, and 35.05% valid. There 
were 156,255 invalid entries of “OOO00”. 


The Outpatient Revenue Code field was 99.91% complete and 79.59% accurate and valid. Of 
the 49,090 Invalid codes, 214 were blank and 48,876 were entries of “0”. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA 
for this validation. The second Diagnosis Code field was 99.94% complete, accurate and 
valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The third 
Diagnosis Code field was 99.87% complete, accurate and valid (incomplete, inaccurate, and 
invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 55.63% complete, accurate and valid. The remaining fields were 
blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code field was 25.53% 
complete, accurate and valid. 


For the Pharmacy claim type, there were 333,870 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. All fields examined were |00.00% complete, accurate and 
valid. 
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What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for HealthCare USA, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. There 
were very few errors encountered in the critical fields examined across all claim types. The 
Outpatient Procedure Code field in the Medical claim type included invalid procedure codes 
(see previous findings); while the Inpatient claim type contained invalid data in the Admission 
Date, Discharge Date, and Patient Status fields. The Revenue Code field contained 63 blank 
entries. For the Outpatient Hospital claim type, the Outpatient Procedure Code and the 
Outpatient Revenue Code fields contained invalid entries. For outpatient claims, a procedure 


code is required only when the Revenue Code is between 300 — 319. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, the rate of Outpatient 

Hospital encounter claim types were consistent with the average for all MC+ MCOs. The rates 
for Dental and Inpatient encounter claim types were significantly higher than the average for all 

MC+ MCOs. This suggests high rates of encounter data submission and good access to 


preventive and acute care. 


To what Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? 

What is the Fault/Match Rate between State encounter Claims and Medical 
Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 


types for the period of January |, 2005 through March 31, 2005 for medical record review. 


Of the 426,225 Medical encounter claim types in the SMA extract file for January |, 2005 
through March 31, 2005, 100 encounters were randomly selected. Providers were requested to 
submit medical records for review. There were 9l medical records (91.0%) submitted for 
review. Encounters for which no documentation was submitted were unable to be validated. 
The match rate for procedures was 42.0%, with a fault rate of 58.0%. The match rate for 


diagnoses was 91.0%, with a fault rate of 9.0%. 
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What Types of Errors Were Noted? 

An error analysis of the errors found on the medical record review for procedure and diagnosis 
codes was conducted. For the diagnosis code in the medical record, the reasons for diagnosis 
codes not matching the SMA extract file were incorrect information (n = 9). The diagnosis 


code listed did not match the descriptive information in the record. 


For the procedure code in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = 41), incorrect codes (n = 12), illegible information (n = |5). Examples of 
missing information included no code, codes listed that were not supported, or codes that did 


not match the procedure description. 


To what extent do the MC+ MCO paid/unpaid encounter claims match the 
SMA paid database? 


Since HealthCare USA included internal control numbers that matched those of the SMA, the 
planned analysis of comparing MC+ MCO encounter data to the SMA encounter claim extract 
file was performed. The SMA defined “unpaid claims’ as those claims that the MCO denied for 


payment, unpaid claims do not include claims paid via a capitation plan. 





For the Pharmacy Claim type, all encounter data submitted to the EQRO (n = 333,870) was of 
“paid” status. There were 33 unmatched claims that were in the HCUSA encounter file and 
absent from the SMA data. Thus, 99.99% of the HCUSA submitted encounters matched with the 


SMA encounter records. 


For all Outpatient Claim Types (Medical, Dental, Home Health, & Hospital; n = 727,244), 79 
“denied” claims were submitted by HCUSA but all other encounter claims were of “paid” status. 
Of the encounter claims submitted by HCUSA, 192 records were unmatched with the SMA 
encounter data. There was a “hit” rate of 99.98% between HCUSA encounter claims and the 


SMA encounter data. 


For the Inpatient Claim Type, HCUSA submitted 53,367 encounter claims. Only 23 of these 
encounter claims were of “denied” status; all other claims were of “paid” status. There were 40 


unmatched records between HCUSA and the SMA, yielding a 99.93% “hit” rate. 
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Why are there unmatched claims between the MC+ MCO and SMA data 
files? 


The majority of unmatched encounters are due to missing ICN numbers which are required to 
match the encounter to that of the SMA. Within the Pharmacy Claim type, 66.67% of the 
unmatched encounters were missing ICN numbers. Therefore, only || Pharmacy claims were 
legitimately missing from the SMA data. For the Outpatient data, 96.88% of the unmatched 
claims were missing ICNs. Of the 6 unmatched claims with ICNs, 2 of those were of “denied”’ 
status and would not be expected to be present in the SMA file. Thus, only 4 unmatched 
encounters were legitimately missing from the SMA data records. For Inpatient Claims, all 


unmatched claims were missing ICNs. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While the MC+ MCO did submit the data in the requested format (including most ICN 
numbers), there are a number of ways to improve the data quality by improving the database 
system. The Internal Control Number is unique only to the encounter but not to each service, 
thus one ICN may be represented in multiple lines of data. To match the MC+ MCO data to 
the SMA data to specific fields, this requires a unique line number. Therefore each service 


provided within an encounter would have a separate line of data with a unique line identifier. 


STRENGTHS 


|. All encounter data was submitted in the specified format and included internal control 
numbers (ICNs) which allowed the EQRO to conduct planned comparisons of the MC+ 
MCO and SMA data files. 


2. The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


3. The critical fields examined for the Dental and Pharmacy claim types were |00.00% 
complete, accurate and valid. 


4. There were significantly higher rates of claims for Dental and Inpatient services than the 
average for all MC+ MCOs. 
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AREAS FOR IMPROVEMENT 


|. For the Medical claim type, there were invalid entries for Outpatient First Date of Service, 
Outpatient Last Date of Service, and Outpatient Procedure Codes. 


2. For the Inpatient claim type, there were invalid entries for the Admission Date, Discharge 
Date and Patient Status, and blank Revenue Code fields. The Revenue Code is a required 
field regardless of the Procedure Code. 


3. For the Outpatient Hospital claim type, there were invalid data in the Outpatient Procedure 
Code and Outpatient Revenue Code fields. 


4. For the comparison of HCUSA encounters with the SMA data file, most unmatched claims 
were due to missing ICN numbers. 


5. The match rate for procedures between the SMA encounter claims extract file and the 
medical records for HealthCare USA were significantly lower than the average for all MC+ 
MCOs. 


RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the 
NSF/CMS 1500 file layout for the Outpatient Procedure Code and run validity checks after 
the programming of new edits. 


2. Ensure that Admission Date, Discharge Date, and Patient Status fields are complete and 
valid for the Inpatient (UB-92) claim types, and institute error checks to identify invalid data. 


3. Always include the Revenue Code regardless of the Procedure Code for the Outpatient 
Hospital and Inpatient claim types (UB-92 layout). 
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8.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract documents with the staff of the Division of Medical Services. On-site review 
time was used to conduct interviews with those who oversee the daily practices of the MCO to 
ensure that documentation is developed and practices occur within the scope of the contract 


and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The Division of Medical Services supplied: 
» State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

= Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection process of actions filed in the first quarter of 2005. 

# 2004 Annual Quality Improvement Program Evaluation 


Additional documentation made available by HealthCare USA included: 
# Marketing Plan and Educational Material Development Policy 
» HCUSA of Missouri Organizational Chart 
# NCQA Quality Improvement Activity Form: Increasing the Rate of Compliance with 
Completion of a Family Therapy Session within 45 days of Initial Outpatient counseling 
Assessment of Members Under the Age of |8 
= Obesity Performance Improvement Project 2006 
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Interviews 
Interviews were conducted with the following group: 
Plan Administration 
Jackie Inglis, VP Health Services 
William Rooney, MD, Medical Director 
Jack Fennig, MD, Medical Director 
Frank Siano, VP Community and Governmental Relations 
Resmi Jacob-Schrieber, Director of Provider Relations 
Pam Victor, Director of Government Relations and Regional Compliance -- Central 
Gene Poisson, Director of Network Development 


Alec Mahmood, CFO 
Deb Fitzgerald, Director of Health Services 


Mental Health 
Vicki Bernard, MHNet 
Susan Norris, MHNet 
Cynthia Williams, MHNet 
Gail Moss, MHNet 
Jackie Inglis, HCUSA 
Gene Poisson, HCUSA 


FINDINGS 

Enrollee Rights and Protections 

A strong commitment to member rights continues to be a cornerstone of HealthCare USA’s 
service philosophy. Quality services to members, with a particular emphasis on families and 
children, were observed within the organization. HealthCare USA views cultural diversity as an 
essential component of their interactions with members. The MCO maintains cultural diversity 
as an aspect of initial and ongoing staff training. HealthCare USA employed staff that spoke 
different languages and is able to provide written materials in languages other than English. 
Maintaining the ability to serve a culturally diverse population with a variety of special service 
needs is exhibited in the MCO’s approach to their work and to their interactions with 


members. 


Healthcare USA initially targeted Early Periodic Screening Diagnosis and Treatment (EPSDT) as 
an area for improvement. Within the past year they have expended considerable energy and 
resources in developing this measure. A plan was instituted to monitor members and produce a 
report on the receipt of the EPSDT screenings two times per year. HealthCare USA sent 
EPSDT fliers as reminders to 6,000 members on a monthly basis. The MCO worked with 
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providers to ensure claims submitted so a record exists of all examinations completed. They 
worked with the community by providing information to Girls and Boys Clubs, after school 
programs, and schools nurses. HealthCare USA met with school boards and districts in both 
the Eastern and Western MC+ Managed Care Program Regions. The objective was to engage 
them in the process of actively ensuring that students receive their health screenings. Efforts 


such as these resulted in a member participation rate of over ninety (90%) percent. 


Ratings of compliance with Enrollee Rights and Protections (69.2%) indicate that HealthCare 
USA made a concerted effort to improve their compliance in this area. The MCO completed a 
number of required policies and that were approved by the SMA. Interviews with administrative 
staff indicated a commitment to attend to the details of completing required policies. The MCO 
had a stated goal of 100% compliance with SMA contract requirements and federal regulations. 
HealthCare USA was encouraged to produce all policy not yet submitted to the SMA, make any 


required revisions in a timely fashion, and to obtain full compliance as quickly as possible. 


Table 66 - Subpart C: Enrollee Rights and Protections Yearly Comparison 













| HealthCare USA 


Federal Regulation 





438.100(a) Enrollee Rights: General Rule 1 2 
438.10(b) Enrollee Rights: Information Requirements 1 1 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 
438.10(c)(4,5) Language and Format: Interpreter Services 1 1 
438.10(d)(1)(i) Information Requirements: Format/Easily 

2 2 
Understood 
438.10(d)(1)(ii)and (2) Information Requirements: Format Visually 1 5 
Impaired, and Limited Reading Proficiency 
438.10(f) Information for All Enrollees: Free Choice, etc. 1 1 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 5 5 
Plans 
438.10(i) Special Rules: Liability for Payment/Cost Sharing 1 i 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee 5 5 
Communications 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 1 2 
438.100(b)(3) Right to Services 1 2 
438.100(d) Compliance with Other Federal/State Laws 2 2 
Number Met 5 9 
Number Partially Met 8 4 
Number Not Met 0 0 


Rate Met 38.5% 69.2% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Behavioral Health 

Staff members from the behavioral health subcontractor MHNet were interviewed at the on- 
site review. The MHNet staff shared information regarding a number of initiatives undertaken 
during 2005. One project involved the support of members through targeted follow-up when 
they are discharged from inpatient treatment. Another measure focused on avoiding weekend 
discharges for members requiring inpatient treatment. MHNet’s goal was to have the member 


ready for discharge prior to Saturday to avoid weekend emergencies. 


The Behavioral Health Organization’s (BHO) system was undergoing enhancement to capture 
baseline information on members receiving behavioral health services. MHNet also made the 
decision to authorized family therapy, in addition to required individual therapy, for all children 
under age 18 who need behavioral health services. The BHO believed that this additional 
resource would assist in ensuring that the family had an understanding of issues facing their 
child, that the entire family would be working together to ameliorate problems, and that the 
family would understand the child’s emotional functioning. The BHO worked closely with 
HealthCare USA to identify expectant mothers to ensure that required behavioral health 
services were in place in an effort to prevent post partum problems. The BHO has also made a 
concerted effort to ensure that information and educational material is translated into different 


languages. Multilingual providers are available to members. 


Quality Assessment and Performance Improvement 


Access Standards 

Healthcare USA worked with both members and providers to ensure proper access to services 
was available. They developed a large provider network throughout all three MC+ Managed 
Care Regions, and continued to recruit providers to expand services available, particularly in the 
Central Missouri area. This enabled members to have an adequate choice for both PCPs and 
specialty providers. The MCO does authorize the use of out-of-network providers when this 


will best meet a member’s healthcare needs. 


One area of concern for Healthcare USA has been locating adequate dental services for 
members, particularly in the Central MC+ Managed Care Region. The dental subcontractor, 
Doral Dental, placed a provider representative in the Central Region to ensure ample 


recruitment occurred and that a representative was available locally. Doral Dental initiated a 
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work plan to obtain additional providers. Healthcare USA Provider Relations worked with 
Doral to ensure that the subcontractor had assistance as needed. Special attention was given to 
the issue of transportation while this network development continues. The MCO paid for 
mileage when a member had a vehicle, or another method of transportation to attend dental 
appointments, when they occurred at an excessive distance. This assisted in increasing the 
availability of services. Another method utilized by the MCO was negotiating an alternative fee 
schedule for providers reluctant to participate due to reimbursement issues. Healthcare USA 
reported that the network did improve, but they continue to concentrate on development 


efforts. 


HealthCare USA reported that the number of requests for PCP changes increased in 2004. The 
MCO obtained information that indicated the increase was partially based on a need for PCP 
accessibility. As a follow-up of a recommendation from the 2004 External Quality Review, the 
MCO implemented a follow-up system, hosted by Provider Relation’s staff, to address a PCP 
change request that appeared to be related to an access problem. The MCO initiated a 
random survey of members as a method of gathering information about the reasons for these 
requests for change. The findings indicated that the greatest number of these requests were 
due to a convenience issue on the part of the member, not because a provider offered 
appointments out of prescribed timeframes. Reasons included a desire to have providers closer 
to the member’s home after a move, a need for providers with extended office hours, and 
changes as the result of auto-assignment. HealthCare USA worked with their providers to offer 
extended hours to meet members’ needs. Several providers have set up “after hours” clinics 
where office space is made available to clinicians willing to work after regular office hours have 
ended. The MCO planned to continue to monitor requests for PCP changes to ensure that 


access to care standards are met. 


Ratings of compliance with Access Standards regulations (64.7%) reflect a number of HealthCare 
USA policies that continue to need submission to or clarification from the SMA. The MCO did 
improve in this area since 2004 and strives to meet all required SMA contract requirements and 


federal regulations. 
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Table 67 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 


HealthCare USA 


Federal Regulation 





438.206(b)(1)(i-v) Availability of Services: Provider Network 
438.206 (b) (2) Access to Well Woman Care: Direct Access 


438.206(b)(3) Second Opinions 


438.206(b)(4) Out of Network Services: Adequate and Timely 
Coverage 


438.206(b)(5) Out of Network Services: Cost Sharing 
438.206(c)(1)(i-vi) Timely Access 

438.206(c)(2) Provider Services: Cultural Competency 
438.208(b) Care Coordination: Primary Care 

438.208(c)(1) Care Coordination: |Identification 
438.208(c)(2) Care Coordination: Assessment 

438.208(c)(3) Care Coordination: Treatment Plans 
438.208(c)(4) Care Coordination: Direct Access to Specialists 
438.210(b) Authorization of Services 

438.210(c) Notice of Adverse Action 


Pe NU N N 
NO NY N_ N 


438.210(d) Timeframes for Decisions, Expedited Authorizations 


438.210(e) Compensation of Utilization Management Activities 


RPeN NIN FIN FF N FIN KN FF 


438.114 Emergency and Post-Stabilization Services 

Number Met 

Number Partially Met 

Number Not Met 0 

Rate Met 52.9% 64.7% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


oO WOAWMO FF NY NY FY FF NY KF NY KF NY N Ff FEF 


Structures and Operation Standards 

HealthCare USA instituted a number of measures to improve practice in this area. The MCO 
held quarterly oversight meetings with all subcontractors in each region to discuss service 
provision issues and to monitor activities. Annual evaluations were completed on each 
subcontractor, and daily contact was maintained. HealthCare USA reported this increased 
contact and monitoring allowed them to address administrative and member issues in a timely 


and effective manner. 


HealthCare USA created a provider advisory group in all three MC+ Managed Care regions. 


They encouraged provider feedback and provided information in a framework that allowed the 
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MCO to develop a true partnership with their provider network. A new method of obtaining 
prior authorizations was made available to HealthCare USA providers. WebMD became 
available to process on-line prior authorizations. Providers began using this method in greater 
numbers. The authorization requests are downloaded at the MCO. It is then the responsibility 


of HealthCare USA to respond in required timeframes. 


Ratings for compliance with Structure and Operation Standards (60%) reflected a lack of 
approved policy and procedures regarding disenrollment and subcontractual relationships. 
Disenrollment policy was submitted to the SMA in December 2005. This policy has not yet 
been approved and contained wording conflicting with the MC+ Managed Care contract. 
Correction and approval of the disenrollment policy, and approval of all policy and requirements 
regarding subcontractors, are required for HealthCare USA to be fully compliant with this 


section of the federal regulations. 


Table 68 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


HealthCare USA 


Federal Regulation 















_ 2004 
438.214(a,b) Provider Selection: Credentialing/Recredentialing 2 2 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 2 2 
438.214(d) Provider Selection: Excluded Providers 2 2 
438.214(e) Provider Selection: State Requirements 2 2 
438.226 and 438.56(b)(1-3) Disenrollment: Requirements and limitations 1 1 
438.56(c) Disenrollment Requested by the Enrollee 1 1 
438.56(d) Disenrollment: Procedures 1 1 
438.56(e) Disenrollment: Timeframes 2 2 
438.228 Grievance System 2 2 
438.230(a,b) Subcontractual Relationships and Delegation 1 1 
Number Met 6 6 
Number Partially Met 4 4 
Number Not Met 0 0 
Rate Met 60.0% 60.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Measurement and Improvement 
The MCO continued to use InterQual as a guide for decision-making in terms of utilization 


review. InterQual criteria were cited when asked about practice guidelines. HealthCare USA’s 
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new medical director, who recently practiced in the Kansas City area was familiar with the 
community based practice guidelines developed there. The MCO expressed the hope that a 
similar group may be formed in the Eastern Missouri MC+ Managed Care Region to explore the 
issue of regionally appropriate practice guidelines. This is an area in which continued 


development is needed by the MCO. 


HealthCare USA continued to have a well developed internal written quality assessment and 
improvement program. The MCO shared their Quality Management Charter and minutes from 
meetings. The Quality Management Program focused on monitoring, assessment, and evaluation 
of clinical and non-clinical service delivery. Initiatives developed as a result of the Quality 
Management Program include EPSDT, the | 7-P program preventing premature delivery, asthma 
prevention program, subcontractor oversight, and the provider advisory group. These quality 
programs targeted members with special healthcare needs, but also provided enhanced services 
to all members. HealthCare USA indicated that they recognized the need to stratify data by 
MC+ Medicaid Managed Care region. The Quality Management charter ensured that meetings 
occur at least quarterly on a regular schedule and had representatives from all sections of the 
organization, as well as including providers. The quality management process ensured that the 


MCO maintained a record of activities, recommendations, accomplishments, and follow-up. 


Through the administrative method, the MCO did report data for Validating Performance 
Measures. HealthCare USA HEDIS data did not include member enrollment dates. This 
omission affected the validity of the data, and compromised the accuracy of the MCO-to-MCO 
comparison. The MCO did submit clinical and non-clinical Performance Improvement Projects. 
The details of the audit are located in the appropriate section of this report. The MCO 
continued to operate a health information system that meets required standards. Encounter 
data was submitted in the format requested so that appropriate validation could occur. The 


details of this process are located in the Validating Encounter Data section of this report. 


Ratings for compliance with Measurement and Improvement regulations (63.6%) reflect the 
inability to appropriately validate Performance Measure data, and the need to develop the 


MCO’s approach to practice guidelines. 
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Table 69 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison 


_ HealthCare USA 


_ Federal Regulation 





438.236(b)(1-4) Practice Guidelines: Adoption 
438.236(c) Practice Guidelines: Dissemination 
438.236(d) Practice Guidelines: Application 


438.240(a)(1) QAPI: General Rules 


438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 
Improvement and PIPs 


438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 
438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 
438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare Needs 
438.240(e) QAPI: Program Review by State NA NA 


NM NN FP N FON WN 
NN FP FPN PF FP WN 


438.242(a) Health Information Systems 2 2 
438.242(b)(1,2) Health Information Systems: Basic Elements 1 2 
438.242(b)(3) Health Information Systems: Basic Elements 1 2 
Number Met 7 7 
Number Partially Met 4 4 
Number Not Met 0 0 
Rate Met 63.6% 63.6% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Grievance Systems 

Rating for compliance with Grievance Systems regulations (100%) indicate that the MCO 
completed all requirements regarding policy and practice in their grievance system. Six member 
files for grievance and appeals were reviewed during the on-site review. All records reviewed 
were handled appropriately and within prescribed time frames. One member grievance 
advanced to a State Fair Hearing where the decision to deny was upheld as the criteria for 
braces was not met. The member files were in order and copies of correspondence were 


included. 


Provider complaints, grievances, and appeals were also reviewed on-site. Seven files were 


requested and all appeared to be in order. When possible, situations were resolved at the 
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complaint level. The provider files were complete with correspondence that met all required 
timeframes. Where appropriate, files contained information that a review occurred by a 
physician who had not been involved in the original decision, or in a previous level of the 
grievance process. HealthCare USA had access to physicians through an independent contract 


who could participate in the review process according to the medical specialty required. 


Table 70 - Subpart F: Grievance Systems Yearly Comparison 





HealthCare 
Federal Regulation | USA 

438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 7. 
438.404(a) Grievance System: Notice of Action - Language and 5 5 
Format 
438.404(b) Notice of Action: Content 2 2 
438.404(c) Notice of Action: Timing 2 2 
438.406(a) Handling of Grievances and Appeals: General 5 5 
Requirements 
438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 
438.408(a) Resolution and Notification: Basic Rule 
438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 
438.408(d)(e) Resolution and Notification: Grievance and Appeals - 

2 2 
Format and Content of Notice 
438.408(f) Resolution and Notification: Grievances and Appeals - 5 5 
Requirements for State Fair Hearings 
438.410 Expedited Resolution of Appeals 2 2 
438.414 Information about the Grievance System to Providers and 5 5 
Subcontractors 
438.416 Recordkeeping and Reporting Requirements 2 2 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 18 18 
Number Partially Met 0 0 
Number Not Met 0 0 
Rate Met 100% 100% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 
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Summary / Follow-up 

HealthCare USA exhibited improvement since the 2004 audit in the completion and submission 
of required policy and procedures to the SMA. The MCO made improvements in compliance in 
three sections of the protocol, maintained 100% compliance in one section, and maintained the 
same rating in one section. The operations and practices revealed during interviews at the on- 
site review indicated a commitment by HealthCare USA to provide quality healthcare services 
to its members. MCO activities focused on enhancing preventative services, creating new 
approaches to providing access to services such as the development of after-hours clinics, 
obtaining member input on issues such as changing PCPs, and responding to prior authorizations 


and grievances in a timely and efficient manner. 


HealthCare USA was not totally compliant in all areas of policy completion and submission. The 
MCO incorporated methods to track required policy submission into daily administrative 
practice and took this process seriously. The practice observed at the time of the on-site 
review provided confidence that the MCO had service to members as their primary focus and 
that there was a commitment to comply with the requirements of the MC+ Managed Care 


contract and federal regulations. 


STRENGTHS 


|. Integration of all specialty sections within the HealthCare USA organization enhances 
the service delivery system. Individuals interviewed could not only speak to their 
specific role at HealthCare USA, but how their work impacts that of other sections and 
how they need to know one another’s focus and projects so they can ensure a holistic 
approach to service delivery. 

2. A commitment to excellence in service delivery throughout the organization was 
observed. Staff at all levels exhibited enthusiasm for ensuring that healthcare services 
provided by HealthCare USA met the needs of all members and were high quality 
services. There was a common expectation that members are treated with respect and 
dignity. 

3. The marketing and outreach plan clearly states and recognizes all components of 
enrollee rights. State requirements are clearly stated and detailed in written material. 

4. HealthCare USA continues to work with Washington University to develop innovative 
approaches to solving problems and providing new alternatives to serve members. 

5. Grievance and appeals files for members were easy to evaluate and followed MCO 
policy. All files contained appropriate notification and actions were timely. 

6. There is recognition by HealthCare USA staff that there are some significant regional 
and cultural differences in the MC+ Managed Care Regions served by the MCO. The 
MCO recognizes the need for diversity within the organization, and in approaches to 
the different communities where services are delivered. 
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AREAS FOR IMPROVEMENT 


|. Completion of all required policies and procedures in a timely manner, to ensure 
compliance with State contract requirements and federal regulations. 

2. Explore development of the Performance Improvement Project process with MHNet to 
better coordinate member services and communication. 


RECOMMENDATIONS 


|. Hold the importance of complying with documentation requirements to the same 
standards as those reflected in the daily practice within the MCO. 

2. Continue MCO development in the area of utilization of available data and member 
information to drive change and support opportunities for organizational growth and 
development. 

3. Continue to track policies and other materials required for annual review. 

4. Continue the commitment to oversight of subcontractors, such as MHNet and Doral 
Dental. Quarterly reviews ensure that member services are at the level the MCO 
required. 
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9.0 - 





9.1 Performance Improvement Projects 


The previous sections of the 2005 EQRO report present the purpose and objectives, 
technical methods, procedures for evaluation, and MCO to MCO comparisons of 
analyses, findings, and recommendations for each of the protocols based on the 
ageregate analysis. This section of the EQRO report summarizes MC+ MCO specific 
methods, procedures, findings, and recommendations for improving the quality, 
timeliness, and access to care for MC+ Managed Care Members. Please refer to the 
main report for detailed technical objectives, methods, and presentation of data that are 


referenced here for the MC+ MCO. 


METHODS 
Document Review 


Missouri Care supplied the following documentation for review: 


# 2005 Attention Deficit Hyperactivity Disorder Performance Improvement Project 


= 2005 Lead Performance Improvement Project 


Interviews 


Interviews were conducted with the project leaders for each Performance Improvement 
Project (PIP) by the EQRO team on March 1|5, 2006 during the on-site review, and 


included the following: 


Tammy Weise — Manager, Quality Management 
Dr. Jan Swaney — Medical Director 

Brent Netemeyer — Manager, Operations 

Katie Dunne — Senior Quality Coordinator 
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The interviewees shared information on the validation methods, study design, and 
findings. Technical assistance regarding study design and presentation of findings was 
provided by the EQRO. The following questions were addressed: 

» Who was the Project Leader? 

= How was the topic identified? 

= How was the study question determined? 

= How can the ADHD study question be simplified to get at the actual issue Missouri Care wants 

to address? 

=» What were the findings? 

=» What was the intervention? 

=» What was the time period of the study? 


» Was the intervention effective? 
» What does Missouri Care want to study or learn from their PIPs? 


The maturity of the PIPs submitted for validation was not complete at the time of the 
initial request for this information in October 2005. The MCO was instructed that they 


could submit additional information that included outcomes of the intervention. 


FINDINGS 
The first PIP evaluated was, “Attention Deficit Hyperactivity Disorder (ADHD) 


Performance Improvement Project.” Originally this PIP was identified as a clinical 
project. This was discussed with the MC+ MCO at the time of the on-site review. The 
PIP was recoded as non-clinical. Although the goal of the project was to improve the 
quality of care for children with ADHD, the methodology and interventions presented 
to achieve this goal were non-clinical. Reviewers and MC+ MCO staff agreed upon this 


change in designation. 


The rationale for identifying this topic of study was well documented in the information 
presented. The narrative related the literature included in the review to the rationale 
for the PIP. It included information on the population and provided a strong argument 
for choosing this topic for a performance improvement project. The overarching goal 
of the project was clearly focused on correcting deficiencies in health care services. To 
accomplish this goal, the PIP planned to implement the utilization of practice guidelines 
and education regarding the identification and quality treatment of children diagnosed 
with ADHD, specifically those children who have been prescribed medication to affect 


their symptoms. All physicians working with MC+ Member children diagnosed with 
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ADHD would be directly impacted. No children were excluded, and this member 


population would be considered as having a special health care need. 


The study question presented was “Does the distribution of ADHD best practice 
guidelines to providers through a provider tool kit and continuing medical education 
(CME) conferences, increase the number of members who ‘are diagnosed and treated 


9 


appropriately.’ The presentation of the study questions, as stated in the project 
narrative, was very complex and actually addressed planned interventions. Simplification 
of the study question was required to identify the actual issue to be addressed and to 


develop a methodology to measure success. 


The definition of each indicator was intricately linked to a related section of the study 
question. The indicators chosen were appropriate, but they added to the complexity of 
the narrative and the study questions, as originally written. This study had tremendous 
potential and should be reworded to ensure that the study question, indicators, and re- 
Survey type questions are clarified. This would allow appropriate documentation and 


data analysis to evaluate the success of the project. 


The identified population was all MC+ Missouri Care Member children age 6-12 who 
have been prescribed a stimulant drug during the study period. The justification for the 
age range was well defined and consistent with NCQA guidelines for assessing the 
quality of care for children with ADHD prescribed medications. No portion of the 
eligible population to be affected was excluded in the study. No sampling was used. 
The study design clearly identified the data to be collected and it documented a 
prospective data analysis plan. A plan for distributing survey tools to providers was 
clearly defined. A Chi-Square analysis was planned to evaluate change for each 
indicator. All claims and pharmacy data to be collected and how it will be analyzed was 


included in the information presented. Survey tools were presented with project 


narrative. 
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All planned interventions were described, but not in sufficient detail to ensure a 
thorough understanding of the rationale presented, or the expected outcomes to be 
achieved by this section of the study. A number of interventions were mentioned, but it 
did not include an anticipated outcome or a method for achieving and understanding the 
results of the intervention. This PIP was not mature enough to achieve compilation of 
data. No results were available. The study did identify post-interventions and a plan for 


continuing to collect data during a latter part of the study. 


lt appears that with proper implementation and data analysis, this study has high 
potential to produce real improvement that will positively impact the intended 
members. The study was developed through the use of a community-based physician 
group who had an interest in impacting this topic. The study design could be refined, 
but the intentions communicated in the narrative indicate a likelihood of significant 


findings. 


The second PIP evaluated was “Lead Performance Improvement Project.” The study 
topic provided was the need to increase the number of members receiving blood lead 
level testing at |2 and 24 months of age. There was a thorough presentation of the 
rationale for selecting this topic for study. The MC+ MCO initiated a performance 
improvement project in 2004 to increase blood lead level testing. The project 
presented built on the foundation of the original PIP. The narrative identified that the 
original intervention produced positive results, but they were not sustained. This PIP 
took a multi-staged approach to intervene in changing physician blood lead level testing 
practices. A pre-test was completed by surveying physicians about blood lead level 
testing practices and perceived barriers. The interventions were then planned around 
the results of the information obtained. The information provided supporting the 
selection of this topic included a literature review and research done by the Kaufman 


Foundation, as well as assessment of State data. 
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The study questions presented were: 


|. Did the proposed interventions result in a significant increase in the number of children | 1-15 months and 
23-27 months who receive blood lead level testing? 

2. Does monitoring physician testing practices and targeting education to physicians with low compliance, 
result in an increase in testing among physicians targeted? 


The study used well-operationalized indicators including a range before and after the 
12" and 24" month birthday. The indicators were clearly tied to the questions on the 
physician survey tool. The information acknowledged that ongoing barriers may 
continue, but the proposed interventions clearly sought to have a positive impact on 
these issues. Data sources were clearly identified including claims and MOHSAIC data. 
A predetermined algorithm was to be utilized in calculating data. All MC+ Members, 
within the definitions of the targeted age groups were included. No members within 
the prescribed age ranges were to be excluded. The two age ranges selected were 
based upon the Missouri Department of Health and Senior Services guidelines. The 
population of targeted providers was clearly defined to include primary care physicians 
and pediatricians in the Missouri Care Health Plan. No sampling was used in this 


project. 


The narrative clearly described the survey information to be obtained from physicians 
and how this data would be analyzed and used. The study included information on how 
the MC+ MCO planned to collect the data on the children targeted. The narrative 
included enough specificity to ensure confidence that this process was thorough and 
complete. Statistical process control charts with a 95% confidence level were used to 
monitor the ongoing process. A Chi-square test was used to compare the percentage 
of children tested in the three months prior to the intervention with the percentage 
tested three months following the intervention. All data sources were clearly defined 
and the prospective data analysis plan was followed. The exact method of matching 


members and providers and testing data reliability were defined and detailed in the 


narrative. 
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There were specific interventions identified in the narrative. How these interventions 
were related to the topic and study questions was evident in the section related to 
provider education. The primary interventions described focused on what will occur 
with providers. The narrative discussion listed a number of interventions targeted at 
member education. These do not appear to be directly related to the study questions 
presented and may be related to the previous PIP that also focused on increasing blood 
lead level testing. The information provided stated that parent follow-through was seen 
as a barrier by physicians. However, this is not the focus of this PIP and impacting this 


issue may not be an appropriate activity for this study. 


Data analysis was not complete. The results of the provider surveys did occur 
according to plan. All data from MOHSAIC was not available for analysis. This PIP was 
well-constructed. As it matures and all data becomes available it has a high potential for 
positive performance improvement. The analysis was planned and the documentation 
provided confidence that this project will be completed as described. The format and 
presentation led to ease in evaluating the project. Information was clear, organized, and 


understandable, all adding to the confidence in the potential outcomes. 


STRENGTHS 


|. Study topics were well defined and addressed issues that will create improved 
services for members. 


2. The format of the Performance Improvement Projects utilized the Conducting PIP 
Protocol. It was easy to follow and enhanced the ability of reviewers to complete the 
validation process. 


3. The studies presented showed promise for having a profound impact on 
performance improvement that can result in quality health care for MC+ Members. 
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AREAS OF IMPROVEMENT 


|. In defining the study topics and study questions, be clear about the intent and 
direction of the study. In both PIPs reviewed, these topics might have been 
considered clinical or non-clinical. The reviewer should be given enough information 
to be able to justify the chosen direction and understand the intent of the stated 
study questions. 


2. When defining measurements, provide enough clarity that the topic, intervention, 
and measurement to achieve the outcomes, all relate to one another. 


RECOMMENDATIONS 


|. Continue to utilize the protocols to evaluate performance improvement studies. 
These studies were not yet mature enough for complete data collection and analysis. 
The validity of the outcomes could not be assessed. 


2. Consider all interventions that may affect the projected outcomes. Ensure that 
there is adequate documentation to explain the impact of the interventions on the 
findings and outcomes. 
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9.2 Validation Of Performance Measures 


METHODS 

Objectives, technical methods, and procedures are described under separate cover. This 
section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Missouri Care. Missouri Care submitted the requested documents 
on December 9, 2005. The EQRO reviewed documentation between December 9 , 2005 and 
February 27, 2005. On-site review time was used to conduct follow-up questions and provide 


feedback and recommendations regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Baseline Assessment Tool (BAT) submitted by Missouri Care 

e MEDSTAT’s NCQA HEDIS Compliance Audit Report for 2005 

e Missouri Care’s HEDIS Data Entry Training Manual 

e Missouri Care’s Policies pertaining to HEDIS rate calculation and reporting 


The following are the data files submitted for review by the EQRO: 


e ADV_DenomAndNumer.xls 
e ADV_EnrollmentData.xls 

e CIS _DenomAndNumer.xls 
e CIS _EnrollmentData.xls 

e CIS _Hybrid.xls 

e WI5 DenomAndNumer.xls 
e WI5_EnrollmentData.xls 

e WI5_Hybrid.xls 


Interviews 

The EQRO conducted on-site interviews with Katie Dunne, Jean Gurucharri, Karen Richards, 
HEDIS Coordinator; Alan Boyett, HEDIS Administrator; and Greg Cohen (of Austin Provider 
Solutions), at Missouri Care in Columbia on Wednesday, March |5, 2006. This group was 
responsible for the process of calculating the HEDIS 2005 performance measures. The 
objective of the on-site visit was to verify the methods and processes behind the calculation of 
the three HEDIS performance measures. This included both manual and automatic processes of 


information collection, storing, analyzing and reporting. 
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FINDINGS 

Missouri Care calculated the Childhood Immunization Status, Combination #2 and Well-Child 
Visits measures using the Hybrid Method. MCO to MCO comparisons of the rates of 
Childhood Immunization Status, Combination #2, Well-Child Visits, and Annual Dental Visit 
measures were conducted using two-tailed z-tests. For comparisons that were statistically 
significant at the 95% confidence interval (Cl), the z-score (z), the upper and lower confidence 


intervals (Cl), and the significance levels (p < .05) are reported. 


The HEDIS 2005 rate for Childhood Immunization Status, Combination #2 reported to the SMA 
and the State Public Health Agency (SPHA) by Missouri Care was 63.66%. This was significantly 
higher than the statewide rate for all MC+ MCOs (28.17%; z = 1.13; 95% Cl: 52.56%, 74.76%; p 
> 95). 


The HEDIS 2005 rate for Missouri Care for the Well-Child Visits measure was 75.69%, which 
was significantly higher than the statewide rate for all MC+ MCOs (38.42%; z = 1.92, statewide 
95% Cl: 63.16%, 88.22%; p > .95). The 2005 HEDIS rate for Missouri Care for the Annual 
Dental Visit measure was 28.66%; comparable to the statewide rate for MC+ MCOs (29.76%, z 
= -0.21; 95% Cl: 24.38%, 32.94%; n.s.). 


The following sections summarize the findings of the process for validating each of the 

performance measures in accordance with the Validating Performance Measures Protocol. The 
findings from all review activities are presented according to the EQRO validation activity, with 
the findings for each measure discussed within the activities as appropriate. Please refer to the 


tables in the main report for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

The information systems (IS) management policies and procedures for rate calculation were 
evaluated consistent with the Validating Performance Measures Protocol. For all three 
measures, Missouri Care was found to meet all criteria for producing complete and accurate 
data (see Attachment V: Data Integration and Control Findings). There were no biases or 
errors found in the manner in which Missouri Care transferred data into the repository used for 


calculating the HEDIS 2005 measures. 
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Documentation of Data and Processes 

Missouri Care and its affiliate, Schaller Anderson, contracted with Austin Provider Solutions 
(APS) for the calculation of the HEDIS 2005 performance measures. The internally-developed 
application had received NCQA certification and been reviewed by NCQA for source code 
validation and efficiency of data integration. The EQRO was provided with a process overview 
of the QMACS claims management system, a registered trademark owned by Quality Care 
Solutions, Inc. (QCSI), and a validation overview of the HEDIS Data repository of APS. The 
EQRO was also provided with an overview of the data flow and integration mechanisms for 
external databases for these measures. Data and processes used for the calculation of measures 
were adequate (see Attachment VII: Data and Processes Used to Calculate and Report 


Performance Measures). Missouri Care met all criteria that applied for all three measures. 


Processes Used to Produce Denominators 

Missouri Care met all criteria for the processes employed to produce the denominators of all 
three performance measures (see Attachment X: Denominator Validation Findings). This 
involved the selection of members eligible for the services being measured. Missouri Care 
employed a 5% oversample rate for the Childhood Immunization Status and Well-Child Visits 
measures. For the Childhood Immunization Status measure, there was one record excluded 
due to contraindications identified through administrative data, and there was one record 


chosen from the auxiliary list for replacement, making for a total sample of 410. 


For the HEDIS 2005 Childhood Immunization Status, Combination #2 measure, the DST 
showed a total of I|,181 eligible members for the denominator. The file of all administrative 
records supplied by the MCO contained I,181 eligible members. There was no duplication of 


members and the dates of birth and dates of enrollment were within the valid range. 


For the HEDIS 2005 Well-Child Visits measure, there were a total of 1,156 eligible members 
listed by the MCO; the EQRO was able to validate 1,150 eligible members. The DST showed a 
denominator of 288, with a final sample size of 303 after a 5% oversample. There were no 
exclusions allowed for the measure, and no exclusions or replacements reported. There were 
no duplicate member names, identification numbers or dates of birth. The dates of birth were 


within the valid range and the dates of enrollment and codes for well care visits were provided. 
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For the HEDIS 2005 Annual Dental Visit measure, there were a total of 4,271 administrative hits 
reported and 4,270 validated. There were no duplicate members and the dates of birth were in 


the valid range. The dates of enrollment were valid. 


Processes Used to Produce Numerators 

All three measures included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2005 criteria 
(see Attachment XIll: Numerator Validation Findings). Medical record reviews were conducted 


for the Childhood Immunization Status and Well-Child Visit measures. 


For the HEDIS 2005 Childhood Immunization Status Combination #2 measure, Missouri Care 
appropriately included administrative events from the State Public Health Immunization Registry 
(MOHSAIC). The DST reported 249 administrative hits from the eligible population; the EQRO 
validated 433 administrative hits, which represents an under reported bias of 27.00%. The MCO 
did not identify to the EQRO which records were selected for the sample, so it is difficult to 
know exactly how many administrative hits were in the sample of 410 records pulled for hybrid 
review. Whereas the MCO reported 94 administrative hits in the sample, the EQRO estimates 
150. For the medical record review validation, the EQRO requested 30 of the 167 medical 
records reported to have contributed to the hybrid hits. A total of 30 of the 30 medical 
records were received for review; 23 of those were validated by the EQRO. Therefore, the 
percentage of medical records validated by the EQRO was 76.67%. The rate calculated by the 
EQRO based on validated administrative and hybrid hits was 67.89%, resulting in an 


underestimate of 4.23%. 


For the HEDIS 2005 Well-Child Visits measure, 197 of the 194 administrative hits reported 
were validated. Of the |,150 eligible members found, the EQRO validated 747 administrative 
hits. The plan reported 712 hits for the total population. The EQRO estimates 197 
administrative hits within the sample, whereas, the MCO reported 194. The EQRO requested 
and received 27 medical records reported to have contributed to the hybrid hits. Of these 27 
medical records, 23 were found to be a “hit”, containing 6 or more well-child visits within the 
first 15 months of life. The remaining records included two (2) members with five well-child 
visits and two (2) members with zero visits. This resulted in a rate of 85.19% of medical records 
validated. The rate calculated by the EQRO based on validated administrative and hybrid hits 


was /5.50%. The rate reported by the MCO was 75.69%, which resulted in an overestimate of 
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0.19% by the MCO. Although the EQRO was only able to validate 85.19% of the hybrid record 
hits reported, the EQRO validated over 100% of the administrative hits reported by the MCO, 


thus resulting in a small overestimate. 


The HEDIS 2005 Annual Dental Visit was the third measure validated. 4,270 of 4,271 
numerator hits reported were validated. The dates of birth, enrollment and services were 
within the valid ranges. The final rate calculated by the EQRO was 28.66%, with no observed 


bias. 


Sampling Procedures for Hybrid Methods 
The Hybrid Method was used for the Childhood Immunization Status and Well-Child Visits 
measures. Attachment XIl; Impact of Medical Record Review Findings and Attachment XV: 


Sampling Validation Findings were completed for each of these measures. 


Submission of Measures to the State 

Missouri Care submitted the DST for each of the three measures validated to the SPHA (the 
Missouri Department of Health and Senior Services; DHSS) in accordance with the Code of 
State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) and the 


SMA Quality Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

Table 7| shows the estimated bias and the direction of bias found by the EQRO. Both 
measures calculated by the Hybrid Method were within the 95% lower confidence limits 
reported by the MCO. There was no bias observed in calculation of the Annual Dental Visit 


measure. 


Table 71 - Estimate of Bias in Reporting of HEDIS 2005 Measures 


Measure Estimate of Bias 
Childhood Immunization Status, Combination #2 Underestimate 


Well-Child Visits in first 15 months of Life Overestimate 
Annual Dental Visit 
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Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all 

data sources that were summarized in the Final Performance Measure Validation Worksheet for 
each measure. Table 7/2 (see below) summarizes Final Audit Ratings based on the Attachments 


and validation of numerators and denominators. 


Table 72 - Final Audit Rating for HEDIS 2005 Performance Measures 

Measure Final Audit Rating 

Childhood Immunization Status, Combination #2 Substantially Compliant 

Well-Child Visits in first 15 months of Life Fully Compliant 

Annual Dental Visit Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by Missouri Care. Not Valid = Measure deviated from State specifications 
such that the reported rate was significantly biased. This designation is also assigned to measures for which no rate 
was reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Missouri Care was fully compliant with the HEDIS 2005 Well Child Visits in the First 15 
months of Life and Annual Dental Visit measure. 


2. Missouri Care demonstrated a significantly higher rate of the HEDIS 2005 Childhood 
Immunization Status and Well-Child Visits measure compared to the average for all MC+ 
MCOs. 


3. Missouri Care demonstrated a higher rate of the HEDIS 2005 Well-Child Visits than both 
the National Medicaid and National Commercial Averages. 


4. Missouri Care demonstrated a higher rate of the HEDIS 2005 Childhood Immunization 
Status than the National Medicaid Average. 


5. There was effective use of data from the State Public Health Immunization Registry 
(MOHSAIC) for the calculation of the immunization rates. Missouri Care provided the 
members’ identification data to the SPHA and obtained an extract of immunizations for the 
members eligible. This data was loaded into the HEDIS data repository (SQL Server) by 
Austin Provider Solutions (APS) and integrated into the calculation of the numerators. 


6. Missouri Care used reporting software by APS for calculation of HEDIS performance 
measures that had been reviewed by NCQA. This application software had also been 
reviewed and tested for source code verification by NCQA. 


7. Missouri Care had no capitated providers, which may have contributed to higher rates of 
administrative hits. 


8. There were effective edit and validity checks within the QMACS claims system. There was a 
mechanism to create error logs within QMACS which enabled the user to create reports 
and verify accuracy. 


9. The relationship that MOCare had with vendors and subcontractors was very open. 


Oseeregee Performance Management Solutions Group 


ba ee animes h al A division of Behavioral Health Concepts, Inc. 310 


Missouri MC+ Managed Care External Quality Review Section 9 
Report of Findings — 2005 Missouri Care Health Plan 


10. MOCare corrected the deficiency from last year’s report and located and archived data 
sheets for medical record review. 


Il. MOCare retrieved 100% of medical records requested for the performance measure 
validation. 


12. MOCare is utilizing statistical significance testing to compare HEDIS rates from year to year. 


13. MOCare has used the 2004 External Quality Review report to improve processes and 
procedures. 


AREAS FOR IMPROVEMENT 


|. Missouri Care used medical record review software, which was used as a tool for retrieving 
medical records that needed to be reviewed. This application had a module for data entry 
that did not indicate whether administrative data had been previously entered. The design 
of this module leaves open the possibility for data entry error by re-entry of valid 
administrative data. Missouri Care follows the process identified by the NCQA auditor, 
MEDSTAT. 


2. Documentation on HEDIS rate calculation and policies related to calculation of HEDIS 
measures are weak at the Columbia office. 


3. QMACS uses 2 systems with 2 different member ID numbers. At present, Missouri Care 
ends all ID’s with an “n” for “new” or an “o” for “old”. This does not seem like the most 
effective way to ensure the uniqueness of ID numbers, therefore Missouri Care should 
consider alternative routes when the system is upgraded. 


4. The HEDIS rate end products are housed at the MOCare office in Columbia, however the 
staff at that office are still working to understand the inputs of the HEDIS calculations. 


RECOMMENDATIONS 


|. Improve documentation of the HEDIS rate calculation process by developing and maintaining 
a set of information system policies for the HEDIS rate production on-site. This will allow 
for continuity and validity of the process of rate production in the event of turnovers. 


2. Increase ownership and control by Missouri Care over the process of calculation of HEDIS 
measures. This can be done by designating an employee to closely coordinate with APS and 
clearly assign responsibilities related to the HEDIS rate calculation processes within the 
organization. 


Continue to conduct and document statistical comparisons on rates from year to year. 


4. Continue to participate in training of MCO staff involved in the oversight of coordination of 
performance measure calculation. 
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9.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical Fields? 


For the Medical claim type, there were 101,758 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 


3. The Outpatient First Date of Service field was 100.00% complete, accurate and 99.99% valid. 
Invalid dates of service ranged from | 2/01/2005 —1 2/27/2005 for six fields. 


4. The Outpatient Last Date of Service field was 100.00% complete, accurate and 99.99% valid. 
Eleven invalid dates of service ranged from 04/01/2005 — 04/29/2005. 


5. The Outpatient Units of Service field was 100.00% complete, accurate and valid. 

6. The Outpatient Procedure Code field was 100.00% complete, accurate and valid. 
7. The Outpatient Place of Service field was 100.00% complete, accurate and valid. 

8. The first Diagnosis Code field was 100.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the 
Health Plan Record Layout Manual, they fell well below the 100% threshold established by 
the SMA for this validation. The second Diagnosis Code field was 18.44% complete, 
accurate and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). 
The third Diagnosis Code field was 3.07% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and 
invalid). The fourth Diagnosis Code field was 0.44% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code 
field was 0.00% (with rounding) complete, accurate and valid. Three fields were complete, 
accurate, and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). 
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For the Dental claim type, there were 2,902 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate 


and valid. 


For the Home Health claim type, there were no encounter claims paid by the SMA for the 


period January |, 2005 through March 31, 2005. 


For the Inpatient claim type, there were | 1,742 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


|. The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Recipient ID field was 100.00% complete, accurate and valid. 

3. The Admission Type field was 100.00% complete, accurate and valid. 
4 


The Admission Date field was 100.00% complete and accurate, 97.24% valid. Invalid 
dates ranged from 02/24/2005 — 12/31/2005. 


5. The Discharge Date field was 100.00% complete with the correct number of characters 
(size). The correct type of information (date format) was present 99.69% of the time 
(with 36 entries of “99999999”). Valid values were present 95.88% of the time. In 
addition to the invalid “99999999” entries, invalid dates ranged from 04/01/2005 — 
04/15/2005. 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Se oS 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% 
threshold for completeness, accuracy, and validity established by the SMA (87.06%, 
70.72%, 53.64%, and 38.79%, respectively). The remaining fields were blank (incomplete, 
inaccurate, and invalid). 


l0. The First Date of Service field was 100.00% complete and accurate, and 97.29% valid. 
Invalid dates of service ranged from 10/10/2005 — 12/31/2005. 


ll. The Last Date of Service field was |00.00% complete and accurate, and 96.18% valid. 
There were four invalid dates of service ranging from 04/01/2005 — 04/15/2005. 


|2. The Revenue Code field was 100.00% complete, accurate, and valid. 


|3. The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were 69,657 encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. Missouri Care had 100.00% complete, 
accurate and valid data for all fields examined, except the third through fifth Diagnosis Codes. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
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Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA for 
this validation. The second Diagnosis Code field was 49.75% complete, accurate and valid. The 
third Diagnosis Code field was 23.10% complete, accurate and valid. The fourth Diagnosis Code 
field was 10.37% complete, accurate and valid. The fifth Diagnosis Code field was 4.95% 
complete, accurate and valid. All remaining Diagnosis Code fields were blank (incomplete, 


inaccurate, and invalid). 


For the Pharmacy claim type, there were 78,685 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. Missouri Care had 100.00% complete, accurate and valid data 


for all fields examined. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Missouri Care, an error analysis of the invalid entries was 
conducted for fields which were lower than the 100.00% threshold specified by the SMA. All 
critical fields for the Outpatient Hospital, Dental and Pharmacy claim types were |100.00% 
complete, accurate, and valid (see previous findings). The Medical Claim Type the Outpatient 
First Date of Service and the Outpatient Last Date of Service fields contained invalid data. The 


Inpatient Claim type had invalid data in the Admission Date and Discharge Date. 


What is the Level of Volume and Consistency of Service? 

When comparing the rate of encounter claim types per 1,000 members, the rates for Medical, 
Pharmacy, Inpatient and Outpatient Hospital claim types were significantly higher than the 
average for MC+ MCOs. The rate for Dental claims was significantly lower than the average for 
MC+ MCOs. This suggests high rates of encounter data submission and access to preventive 


and acute care. 


To what Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? 

What is the Fault/Match Rate between State encounter Claims and Medical 
Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 


types for the period of January |, 2005 through March 31, 2005 for medical record review. 
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Of the 101,758 Medical encounter claim types in the SMA extract file for January |, 2005 
through March 31, 2005, 100 encounters were randomly selected. Providers were requested to 
submit medical records for review. There were 100 medical records (100.0%) submitted for 
review. Encounters for which no documentation was submitted were unable to be validated. 
The match rate for procedures was 71.0%, with a fault rate of 29.0%. The match rate for 


diagnoses was 100.0%, with a fault rate of 0.0%. 


What Types of Errors Were Noted? 
An error analysis of the errors found in the medical record review for procedure codes was 


conducted. There were no errors found in the medical record review for diagnoses codes. 


For the procedure codes in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = |5), incorrect codes (n = |4). Examples of missing information included no 
code, codes listed that were not supported, or codes that did not match the procedure 


description. 


To what extent do the MC+ MCO paid/unpaid encounter claims match the 
SMA paid database? 


Since Missouri Care included internal control numbers that matched those of the SMA, the 
EQRO conducted the planned analyses comparing MC+ MCO encounter data to the SMA 
encounter claim extract file. The SMA defined “unpaid claims” as those claims that the MCO 


denied for payment, unpaid claims do not include claims paid via a capitation plan. 





For the Pharmacy Claim type, all encounter data submitted to the EQRO (n = 78,685) was of 
“paid” status. There were 35 unmatched claims that were in the MOCare encounter file and 
absent from the SMA data. Thus, 99.96% of the EQRO submitted encounters matched with the 


SMA encounter records. 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 315 





Missouri MC+ Managed Care External Quality Review Section 9 
Report of Findings — 2005 Missouri Care Health Plan 


For all Outpatient Claim Types (Medical, Dental, and Hospital), MOCare submitted 174,317 
“paid” encounters. Of these encounter claims 214 records did not match with the SMA 
encounter claim extract file. There was a “hit” rate of 99.88% between MOCare encounter 


claims and the SMA encounter data. 


For the Inpatient Claim Type, MOCare submitted | 1,742 encounter claims. All encounter 
claims were of “paid” status. 79 encounters from MOCare were not matched in the SMA 


extract file, yielding a 99.33% “hit” rate. 


Why are there unmatched claims between the MC+ MCO and SMA data 
files? 


The majority of unmatched encounters are due to missing ICN numbers which are required to 
match the encounter to that of the SMA. Within the Pharmacy Claim type, 34.29% of the 
unmatched encounters were missing ICN numbers. Therefore, 23 Pharmacy encounter claims 
were legitimately missing from the SMA extract data. For the Outpatient data, 85.58% of the 
unmatched claims were missing ICNs. Thus, there were 33 unmatched encounters that were 
legitimately missing from the SMA data records. For Inpatient Claims, all unmatched claims 


were missing ICNs. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While the MC+ MCO did submit the data in the requested format (including most ICN 
numbers), there are a number of ways to improve the data quality by improving the database 
system. The Internal Control Number is unique only to the encounter but not to each service, 
thus one ICN may be represented in multiple lines of data. To match the MC+ MO data to the 
SMA data to specific fields, this requires a unique line number. Therefore each service provided 


within an encounter would have a separate line of data with a unique line identifier. 
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STRENGTHS 


|. Encounter data was submitted to the EQRO in the requested format and even included 
internal control numbers which enabled BHC to conduct the planned comparisons between 
the MC+ MCO and the SMA extract files. 


2. The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


3. The critical fields examined for the Dental, Outpatient Hospital, and Pharmacy claim types 
were 100.00% complete, accurate and valid. 


4. In the Inpatient Claim type, Patient Status and Units of Service fields increased to 100.00% 
complete, accurate, and valid from 2005. 


5. The rates for Medical, Pharmacy, and Outpatient Hospital claim types were significantly 
higher than the average for MC+ MCOs, suggesting high rates of encounter data submission 
and at least moderate access to preventive and acute care. 


6. Revenue Code fields were complete and valid for all Outpatient Hospital claim type 
encounters. 


7. The rates of identification of administrative hits for performance measures were significantly 
higher for Missouri Care, suggesting more complete claims data for immunizations and well- 
child visits. 


8. Missouri Care had significantly higher rates of match for the procedure and diagnosis codes 
between medical records and the SMA encounter claims data than the average for all MC+ 
MCOs. This was largely accounted for by the higher rate of medical records submitted by 
providers. 


AREAS FOR IMPROVEMENT 


|. The Outpatient First Date of Service and Outpatient Last Date of Service fields contained 
invalid entries for the Medical claim type. 


2. The Admission Date and Discharge Date fields had invalid entries for the Inpatient claim 
type. 


RECOMMENDATIONS 


|. Ensure that Admission Dates and Discharge Date fields are complete and valid for the 
Inpatient (UB-92) claim types, and institute error checks to identify invalid data. 


2. Include all State issued ICN numbers for all encounters to allow more accurate matching of 
encounters between the MC+ MCO and SMA extract files. 
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9.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract documents with the staff of the Division of Medical Services. On-site review 
time was used to conduct interviews with those who oversee the daily practices of the MCO to 
ensure that documentation is developed and practices occur within the scope of the contract 


and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The Division of Medical Services supplied: 
» State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

= Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection process of actions filed in the first quarter of 2005 

# 2004 Annual Quality Improvement Program Evaluation 


Additional documentation made available by Missouri Care Health Plan included: 
#» Marketing Plan and Educational Material Development Policy 
# Missouri Care Organizational Chart 
= Participating Health Provider Agreements 
» Missouri Care Policy Tracking Log 
» Missouri Care Provider Directory 
= Lead Screening Initiative Handouts and Informational Packet 
# Missouri Care ADHD Toolkit 
# Missouri Care Informational Handouts 


Performance Management Solutions Group 
eer A division of Behavioral Health Concepts, Inc. 318 





Missouri MC+ Managed Care External Quality Review Section 9 
Report of Findings — 2005 Missouri Care Health Plan 


Interviews 
Interviews were conducted with the following groups: 
Plan Administration 

Susan Christy, Health Plan Administrator 

Dr. Jan Swaney, Medical Director 

Melody Dowling, UM Manager 

Tammy Weise, Manager, Quality Management 


Brenda Moore, Manager, Medical Management 
Debby Langley, Manager, Member Solutions 


Mental Health 
Dr. Jan Swaney, Medical Director 
Melody Dowling, UM Manager 
Tammy Weise, Manager, Quality Management 
Brenda Moore, Manager, Medical Management 
Debby Langley, Manager, Member Solutions 


FINDINGS 

Enrollee Rights and Protections 

Missouri Care had an assigned compliance officer who maintained a record of all internal policies 
and presented reminders to appropriate staff when annual reviews were required. Compliance 
reviews were conducted every other month. Records included all initial approval dates to 
ensure that timely monthly reminders were produced. Revisions were made as necessary. 
Internal approval included the Quality Management Oversight Committee, Managers, the 
Medical Director, and the Chief Executive Officer or Plan Administrator prior to submission to 
the SMA. After this process was complete, the policies were presented to the MCO’s Board of 


Directors. 


The MCO continued to utilize the Child and Adolescent Health Measurement Initiative 
(CAHMI) survey instrument for member needs assessment. Missouri Care utilized the monthly 
special needs listing produced by the SMA and sent the survey to all of their members appearing 
on this listing. If they received no response in seven days, and again in fourteen days, they made 
additional attempts using telephone contacts. If the MCO was unable to contact the member 
after 30 days, the file was closed. Missouri Care reported they send out 75-100 CAHM|Is each 
month and have a 30-35% response rate. The MCO found that by using the CAHMI, it assisted 


in correctly identifying members who needed physical or mental health case management 


services. 
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Missouri Care participated in community-based programs throughout their MC+ Medicaid 
Managed Care region. They were involved in school-based health clinics with the Morgan 
County School District. The MCO participated in a back-to-school fair where they not only 
contacted member families directly, but were able to network with regional primary care 
physicians (PCPs). Additionally, outreach calls were made to all eligible children. Seventy 
EPSDT examinations were performed as the result of these efforts. Missouri Care partnered 
with another MCO to do school based screenings with the School of the Osage. Fifty EPSDT 
examinations and fifteen referrals for follow-up services occurred as a result of this outreach 
effort. Two children were reported to the Child Abuse and Neglect Hotline. One local 
Federally Qualified Health Center (FQHC) held Thursday evening appointments to do Pap tests 
and adolescent EPSDT examinations. As a trial intervention, Missouri Care scheduled 
appointments for the FQHC utilizing demographic information obtained from their system. 
Theses efforts resulted in 46 Pap tests and fourteen EPSDT examinations. An additional 550 
EPSDT examinations followed, after sending informational postcards and informational fliers to 
those members whose tests were past due. Through efforts with the Columbia Public Schools, 
the MCO targeted a campaign to increase EPSDT examinations in the Boone County section of 
the region. EPSDT examinations for high school students were planned at the new Family 
Health Clinic satellite location near Douglas High School. A quarterly news letter for school 


nurses was developed and distributed by the MCO. 


During 2005 Missouri Care also attempted a cervical cancer screening initiative with the FQHC. 
All eligible members received reminder letters and follow-up telephone contacts. A larger 
sample received an informational flier that included a list of providers who could perform these 
examinations. One hundred sixty-one members obtained well-women examinations in a three 
month period following these efforts. The MCO considered this a successful initiative in terms 


of member response. 


The rating for Enrollee Rights and Protections (84.6%) reflects that the MCO substantially 
complied with the submission and approval of all policy and procedures to the SMA. All 
practice observed at the on-site review indicated that the MCO appeared to be fully compliant 


with MC+ Medicaid Managed Care Contract requirements and federal regulations in this area. 
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Table 73 - Subpart C: Enrollee Rights and Protections Yearly Comparison 







Missouri Care 


Federal Regulation 





438.100(a) Enrollee Rights: General Rule 1 2 
438.10(b) Enrollee Rights: Information Requirements 2 2 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 
438.10(c)(4,5) Language and Format: Interpreter Services 1 2 
438.10(d)(1)(i) Information Requirements: Format/Easily 

2 2 
Understood 
438.10(d)(1)(ii)and (2) Information Requirements: Format Visually 5 5 
Impaired, and Limited Reading Proficiency 
438.10(f) Information for All Enrollees: Free Choice, etc. 2 2 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 5 5 
Plans 
438.10(i) Special Rules: Liability for Payment/Cost Sharing 1 1 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee Communications 2 2 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 1 2 
438.100(b)(3) Right to Services 1 1 
438.100(d) Compliance with Other Federal/State Laws 1 2 
Number Met 7 11 
Number Partially Met 6 2 
Number Not Met 0 0 


Rate Met 53.8% 84.6% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Behavioral Health 

Through efforts with the SMA, the University of Missouri, and other State agencies, Missouri 
Care had tele-psychiatry services available in six counties in the Central Missouri region. Access 
was made available in outpatient offices for use by specialist psychiatrists. Face-to-face sessions 
with the member’s behavioral health provider were required. Pediatric and adolescent 
psychiatrists were made available through this method in outlying counties, where these services 
would normally be unavailable. In some cases, the parent and case manager participated in 
sessions with the member and psychiatrist. This innovation created a more comprehensive 


approach to treatment for a number of members. 


Missouri Care began supervising the provision of behavioral health services themselves during 
2005. In preparation for this transition, the MCO contacted all providers contracted with 


CommCare in the Central Missouri MC+ Medicaid Managed Care Region. Most active 
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providers were recruited directly into the Missouri Care system. Missouri Care utilized Mid- 
Missouri Mental Health, Boone Hospital, Audrain Medical Center, Royal Oaks Hospital, and St. 
Mary’s Hospital for inpatient behavior health treatment. At the time of the on-site review, the 
MCO was completing a contract with Preferred Family Healthcare, which is based in Kirksville, 
but had centers in several Central Missouri counties. Missouri Care completed provider 


training for the transition of behavioral health services in August 2005. 


Prior authorizations for outpatient behavioral health services were faxed or telephoned to the 
MCO’s main office, and were handled similarly to medical prior authorization requests. 
Requests for inpatient services were called in to the MCO. The Level of Care Utilization 
System (LOCUS) or Child and Adolescent Level of Care Utilization System (CALOCUS) criteria 
were utilized for decision-making criteria. Scoring was completed by nursing staff using 
information submitted by the behavioral health providers and by physicians involved with the 
member. The request, with the corresponding score, was then sent to the Chief Medical 


Officer. Consultation with the Chief Medical Officer for Mental Health occurred as required. 


Accessibility to behavioral health services was enhanced for Central Missouri members by 
incorporating the prior authorization and daily utilization management systems into the MCO’s 
regular operations. Missouri Care found that when an inpatient stay is necessary, information 
was available and the decision-making process was performed quickly and more efficiently. Data 
was available at the MCO and could be processed immediately. The MCO reported that it had 
a clearer understanding of member needs, which led to the most effective levels of care for 
members. Missouri Care reported that inpatient hospitalizations decreased during 2005. One 
reason given was that Royal Oaks Hospital implemented an inpatient screening procedure for 
the MCO. Missouri Care believed this process reduced inappropriate admissions, although it 


did create a slight increase in outpatient services. 


Case management for mental health was done by a master’s level social worker and staff under 


the direction of the Chief Medical Officer for Behavioral Health. 
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Quality Assessment and Performance Improvement 


Access Standards 
Prior authorization policy was reviewed and approved by the SMA during 2005. All prior 


authorizations were faxed or called in to nurses, who screened requests using Milliman Criteria. 
Requests that did not appear to meet the criteria initially were forwarded to the Chief Medical 
Officer for final determination. Decisions were then returned to the nurses who submitted 
notification to the provider. If the MCO received an appeal of a denial, the medical information 
was reviewed by the corporate medical officer for a final decision. The Chief Medical Officer 


for Behavioral Health provides expertise in the review of requests for these services. 


The Missouri Care Nurse Line call center, located in Phoenix, Arizona, is staffed 24 hours per 
day, seven days per week. Both nurse and physician coverage is available. The MCO reported 
that when behavioral health services moved to Missouri Care, they were able to transfer the 
previous CommCare toll-free telephone number for member usage. This enabled the 
transition to remain as seamless to members as possible. All calls received in Phoenix are 


tracked to the Columbia office the next day. 


The MCO integrated physical and mental health services including prior authorizations, the 
Nurse Line, Provider Relations and Member Solutions. Missouri Care was in the process of 
integrating the physical and mental health case management information into one database at the 


time of the on-site review. 


Missouri Care began to use predictive modeling to assist in the identification of members for 
case management. The system was only used for physical health case management during 2005. 
The process assisted the MCO in identifying members who needed case management, but who 
had not previously come to their attention. The MCO planned to incorporate non-compliance 
with medical recommendations into the logic of this program in the future. The language for 
care planning was being incorporated into all Missouri Care policy. The care plans were to be 
developed by the provider and member, with the assistance of the MCO case manager as 


needed. Provider education was completed in 2005. 


Missouri Care reported that their dental subcontractor did continue to search for additional 


providers throughout the Central Missouri region. Dentists were added throughout 2005, but 
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recruitment continued. The MCO added the University of Missouri — Kansas City Dental 


School and several mobile dental units to their network. 


The rating for Access Standards (84.6%) indicated the MCO has actively worked toward 
becoming fully compliant with all MC+ Medicaid Managed Care requirements and federal 
regulations. All practice in this area observed at the time of the on-site review indicated that 
Missouri Care worked toward ensuring that members have access to all the healthcare services 


that they may require. 


Table 74 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 





: Missouri Care 
Federal Regulation 





438.206(b)(1)(i-v) Availability of Services: Provider Network 2 2 
438.206 (b) (2) Access to Well Woman Care: Direct Access 2 2 
438.206(b)(3) Second Opinions 2 2 
438.206(b)(4) Out of Network Services: Adequate and Timely 5 5 
Coverage 

438.206(b)(5) Out of Network Services: Cost Sharing 2 2 
438.206(c)(1)(i-vi) Timely Access 2 2 
438.206(c)(2) Provider Services: Cultural Competency 2 2 
438.208(b) Care Coordination: Primary Care 2 2 
438.208(c)(1) Care Coordination: !Identification 2 2 
438.208(c)(2) Care Coordination: Assessment 1 2 
438.208(c)(3) Care Coordination: Treatment Plans 2 2 
438.208(c)(4) Care Coordination: Direct Access to Specialists 2 2 
438.210(b) Authorization of Services 1 2 
438.210(c) Notice of Adverse Action 1 2 
438.210(d) Timeframes for Decisions, Expedited Authorizations 1 1 
438.210(e) Compensation of Utilization Management Activities 2 2 
438.114 Emergency and Post-Stabilization Services 1 1 
Number Met 12 15 
Number Partially Met 5 2 
Number Not Met 0 0 
Rate Met 70.6% 88.2% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Structures and Operation Standards 

The ratings for Structure and Operations (90%) reflect full compliance with the MC+ Medicaid 
Managed Care contract requirements and federal regulations. The MCO submitted all required 
policy for approval, and all practice observed at the time of the on-site review indicated 
compliance in this area. All credentialing policy and practice was in place. All disenrollment 


policy was complete and all subcontractual requirements were met. 


Table 75 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


Missouri Care 


Federal Regulation 





438.214(a,b) Provider Selection: Credentialing/Recredentialing 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 
438.214(d) Provider Selection: Excluded Providers 


438.214(e) Provider Selection: State Requirements 


438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 
limitations 


438.56(c) Disenrollment Requested by the Enrollee 
438.56(d) Disenrollment: Procedures 

438.56(e) Disenrollment: Timeframes 

438.228 Grievance System 

438.230(a,b) Subcontractual Relationships and Delegation 


Number Met 
Number Partially Met 
Number Not Met 
Rate Met 90.0% 90.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Or OO NY NN NN N NN N F&F N 
Or OO NY NN NN N NY N Ff N 


Measurement and Improvement 

Missouri Care operated a Quality Management Oversight Committee made up of the Chief 
Executive Officer, Plan Administrator, Chief Medical Officer, and department managers. The 
goal of this group was to provide oversight of all operations and MCO initiatives. The MCO 
began an initiative to bring PCPs and psychiatrists together through their pharmacy service to 
look at polypharmacy issues. Pharmacy data was reviewed and members with fifty or more 
prescriptions within a six month period were identified. Letters were sent to all associated 


providers with the date of care and the prescription with drug category. A one page survey was 
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sent out to providers to obtain feedback in August 2005. The plan was to run the project for 


one year to see if the number of prescriptions per member was impacted. 


Acting on a concern for the care of children with ADHD, Missouri Care developed an 
informational packet for PCPs. They reported the packets were well received, one clinic 
requested additional packets to share with residents. The Vice Chairman of Pediatrics at the 
University of Missouri agreed to champion the project with physicians, and to share this 


information during CME presentations. 


The MCO adopted and disseminated practice guidelines in the area of diabetes, asthma, chronic 
obstructive pulmonary disease (COPD), and congestive heart failure. This information was 
available to all providers on the MCO website. Missouri Care indicated that they were in the 


process of developing practice guidelines for depression management. 


The MCO submitted two Performance Improvement Projects (PIPs), which included enough 
information to complete validation. All Performance Measurement data and medical records 
requested were submitted for validation within requested timeframes. Missouri Care did have a 
health information system (HIS) capable of meeting the MC+ Medicaid Managed Care program 
requirements. The MCO also submitted all required encounter data in the format requested. 


The specific details can be found in the appropriate sections of this report. 


The rating for the Measurement and Improvement section (63.6%) reflects that all required 
policy and procedure had been submitted to the SMA for their approval. It appeared that all 
practice observed at the time of the on-site review met the requirements of the MC+ Medicaid 


Managed Care contract and the federal regulations. 
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Table 76 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison 


Missouri Care 


Federal Regulation 





438.236(b)(1-4) Practice Guidelines: Adoption 2 
438.236(c) Practice Guidelines: Dissemination 2 
438.236(d) Practice Guidelines: Application 2 
2 
1 


438.240(a)(1) QAPI: General Rules 


438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 
Improvement and PIPs2 


438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 


1 
438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 2 2 
2 


438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare 
Needs 


438.240(e) QAPI: Program Review by State NA NA 
438.242(a) Health Information Systems 2 2 
438.242(b)(1,2) Health Information Systems: Basic Elements 1 1 
438.242(b)(3) Health Information Systems: Basic Elements 1 1 
7 7 
4 4 


RFP NUN N N_ N 


Number Met 

Number Partially Met 

Number Not Met 0 0 
Rate Met 63.6% 63.6% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; I= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Grievance Systems 

A pattern of claim denials for “duplicate claim line” was noted for Missouri Care. They 
explained that when duplicate procedures occurred for the same member, the same claim 
number was assigned by the SMA claim system. The MCO’s health information system kicked 


these claims out as duplicates. 


Five member appeals and three member grievances were reviewed. Two of the member 
appeals were reviewed and overturned, while three were upheld. Two of the member 
grievances concerned drivers with the transportation subcontractor, Medical Transportation 


Management (MIM). The drivers in these cases were counseled about their treatment of 
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members and their timeliness. The final grievance concerned dental care. It was investigated 
and no quality of care issues were identified. The member failed to appear at a follow-up 


appointment. 


Six provider complaints were reviewed. Three of these were for claim denials where no prior 
authorization had been requested. These were reviewed and upheld. Two complaints stated 
claims had not been paid. Upon review it was found that one of the claims had been paid and 
the other had been billed incorrectly, therefore the denial was upheld. The final complaint 
concerned a denial for a requested drug. Additional information was provided and the denial 


was overturned. 
All files reviewed were in order and all correspondence was dated according to policy timelines. 


The rating for Grievance Systems (100%) reflects that all policy and practice met the 


requirements of the MC+ Medicaid Managed Care contract and federal requirements. 
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Table 77 - Subpart F: Grievance Systems Yearly Comparison 





———E———————— ee Missouri Care 
Federal Regulation 


438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2 
438.404(a) Grievance System: Notice of Action - Language and 5 5 
Format 

438.404(b) Notice of Action: Content 2 2 
438.404(c) Notice of Action: Timing 2 2 
438.406(a) Handling of Grievances and Appeals: General 5 5 
Requirements 

438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 

438.408(a) Resolution and Notification: Basic Rule 

438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 

438.408(d)(e) Resolution and Notification: Grievance and Appeals - 5 5 
Format and Content of Notice 

438.408(f) Resolution and Notification: Grievances and Appeals - 5 5 
Requirements for State Fair Hearings 

438.410 Expedited Resolution of Appeals 1 2 
438.414 Information about the Grievance System to Providers and 5 5 
Subcontractors 

438.416 Recordkeeping and Reporting Requirements 2 2 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 17 18 
Number Partially Met 1 0 
Number Not Met 0 0 
Rate Met 94.4% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 


Summary / Follow-up 

Missouri Care made significant improvement in meeting all policy, procedure, and practice 
requirements to be in compliance with the MC+ Medicaid Managed Care contract and the 
federal regulations. The MCO utilized the tools produced by the 2004 External Quality Review 
as guidelines in ensuring that required written materials were submitted to the SMA in a timely 
and efficient manner. The staff within Missouri Care exhibited a commitment to quality and 
integrity in the work with their members. The MCO utilized unique processes, such as bringing 


the provision of behavioral health services into the organization, as a method for improving the 
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access, quality and timeliness of member services. Missouri Care created tools to educate and 
inform the community and providers, evidenced by the efforts made to improve EPSDT 
examination numbers. The MCO demonstrated an attitude of respect toward their members in 
a number of outreach initiatives, as well as efforts to utilize software tools to better identify 
special healthcare needs. Missouri Care attempted to create a healthcare service system that 
was responsive and assisted members in overcoming the barriers they encounter in a largely 


rural area. 


STRENGTHS 


|. Utilization of both the State Contract Compliance tool and the EQRO tool to measure and 
track compliance with regulations and required policy submissions. 

2. Tracking the submission and approval of required policy. 

3. Transferring the provision of behavioral health services to Missouri Care. The work done 
to complete this transition was seamless for members. The MCO reported evidence that 
the provision of behavioral health services and case management was enhanced by their 
direct responsibility for this aspect of member services. 

4. The dedication and enthusiasm of the medical director and all MCO staff interviewed. 

5. Enhanced interdepartmental communication. 


AREAS FOR IMPROVEMENT 


|. Ensuring that the most current policy is submitted in a timely manner and that all tracking 
tools are up-to-date. 

2. Continued attention to ensure that members have access to an adequate dental care 
network. 


RECOMMENDATIONS 


|. Continue MCO development in the area of utilization of available data and member 
information. This will drive change and create opportunities for further service 
development. 

2. Continue working with school districts throughout the Central Region to engage their 
support and interest. 

3. Continue monitoring access to dental care and assist in recruitment of providers 
throughout the Central Missouri Region. 
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10.1 Performance Improvement Projects 


The previous sections of the 2005 EQRO report present the purpose and objectives, technical 
methods, procedures for evaluation, and MCO to MCO comparisons of analyses, findings, and 
recommendations for each of the protocols based on the aggregate analysis. This section of the 
EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed 
Care Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


METHODS 


Document Review 
Children’s Mercy Family Health Partners supplied the following documentation for review: 
¢ Improving Lead Screening for 6-36 Months Performance Improvement Project 


¢ Access to Primary Care Services Performance Improvement Project 


Additionally the MC+ MCO supplied data at the time of the on-site review, as promised with 


the original data submission. 


Interviews 

Interviews were conducted with the project leaders for each Performance Improvement Project 
(PIP) by the EQRO team on March 8, 2006 during the on-site review, and included the 
following: 


Ma’ata Touslee — Director, Health Services 
Jenny Hainey — Manager, Quality Management 
Lisa Gable — Manager, Health Services 
Augusta Amadi — Case Manager 

Melody Derks 


Johanna Groves 
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Interviewees shared information on the validation methods, study design, and findings. Technical 
assistance regarding study design and presentation of findings was provided by the EQRO. The 
following questions were addressed: 

¢ What activities were added to this project for 2005? 

¢ Was the population for the study expanded? 

@ How were the accuracy, consistency, and validity assured? 

@¢ Why was the lead project valid for continuation and used as a PIP for this project year? 

¢ What findings were relevant to the MC+ population? 

¢ How was improvement analyzed? 


@¢ What are the conclusions about the effectiveness of the interventions analyzed? 


FINDINGS 

The first PIP evaluated was “Improving Lead Screening at 6-36 Months.” The MC+ MCO has 
conducted focused studies on the topic of rates of lead screening since 1999. Their annual 
analysis indicated that the blood lead level testing rates need to be higher. The topic of this PIP 
was designed to increase the percentage of members screened. The intervention was the use of 
provider and member education about the importance of lead screening. New activities 
included the provider education tools and face-to-face visits with area health departments to 
educate staff about the importance of blood lead level screening and the need to bill the MC+ 
MCO to ensure that proper records were available for MC+ Members. Additional study groups 
were added to the member activities. The topic was explained and supported with local, state, 
and national statistics. It did include a literature review and references. The MC+ MCO also 
conducted a comprehensive review of factors contributing to the problem during the previous 


year, which assisted in defining the parameters of the current study. 


The study population was expanded to include all eligible children. No specific group was 
excluded. The study included two questions: 
¢ Do letters and reminder calls to children identified as needing blood lead testing result in 
increased lead testing for those individuals? 
¢ Do letters and reminder calls to children identified as needing blood lead level testing result in 


increased referrals to case management for individuals with high lead levels? 
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Both questions were designed to relate to specific indicators and interventions in the study. 
Indicators included rates of blood lead level testing per member of the study population, with a 
goal of reaching 75; and the rate of case management cases referred due to blood lead levels 
greater thanl0 units/dl. This information was queried every three months through the case 
management data base. These indicators were well defined and included a methodology for 
analysis in the study plan. Initial results indicated a significant increase in blood lead level testing 
that measured the changes in health status of MC+ Members. All eligible members were 


included and the definition of the included population was explained in the narrative. 


The study design clearly specified the data to be collected in the study. The methodology 
prescribed utilized and provided analysis of all required information. This included claims data 
and analysis of the case management database. The project utilized quarterly scans of all 
specified data sources to ensure that a systematic and reliable method to collect required 
information occurred. Interventions included letters to members, presentations to and work 
with providers, and work with local health departments. The data analysis plan calls for 
quarterly queries of the claims data for the study population. Additionally, all members that the 
MC+ MCO is unable to contact will be placed in a control group to assist in evaluating the study 
effectiveness. Pre and post-intervention effectiveness will be measured. An analysis was 
performed as prescribed in the project planning. Preliminary findings were provided and 
updated at the on-site review. Graphs were included, but they were not clearly labeled. 
Additional clarification including an explanation of information presented would be helpful. 
Analysis included a comparison of members who received blood lead level testing before and 
after the interventions and correction for those who received tests prior to the beginning of the 
intervention. Real number and percentages were presented. The significance was identified and 
the actual changes were identified in the numbers available for review. A thorough analysis 


occurred according to the prospectively defined plan. 


The 2005 results indicated a dramatic improvement in the number of children in the defined age 
groups that received blood lead level screenings. Increased numbers of members obtaining case 
management services was also captured and presented in this study. The MCO also contacted 
members as described in the post-intervention plan. The results of this activity supported the 
assumption that the multifaceted approach presented in this study resulted in the positive effects 


that were presented. 
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The narrative, documentation presented, and follow-up information indicated that this study 
provides a high confidence level and that this performance improvement project had the positive 
effect on member behavior sought. The information also indicates that the approaches 
described will be implemented into daily operations of the MC+ MCO to achieve ongoing and 


sustained improvement. 


The second PIP evaluated was “Improved Access to Primary Care Services.” The study topic 
was explained in detail and justified in the narrative and additional documentation provided. The 
presentation was generally based on the MC+ MCO’s previous findings and project plan. This 
information was not based on any specified information or literature review. This project was 
also generated by the previous year’s attempts to impact inappropriate use of emergency room 
services. The presentation is very well developed, but could have been enhanced by conducting 


a literature review and citing research conducted on the national level. 


The project was clearly focused on correcting deficiencies in MC+ Member health care. It was 


based on the following hypothesis: 


Use of the emergency room for non-emergent needs, as a choice over use of the assigned primary care physician 
(PCP), is not supporting the members’ medical needs; and 
Contact with and assistance to members in accessing their PCP or urgent care center will decrease the inappropriate 


use of the emergency room and strengthen PCP relationships. 


The project was open to all members using the emergency room for non-emergent medical 
services. However, the description of the population included no specifics, including 
demographics of MC+ Members who utilized emergency room for non-emergent services in the 
past. This information may have enhanced the justification of the topic and the focus on 
prevention. The focus of the study appears to be on the adult population as the intervention 
takes place at Truman Medical Center which focuses on adult services. The study questions for 
this topic posed in the documentation were: 

¢ Does placing a case manager in the emergency room (ER) during peak hours for education of 

members reduce overall ER utilization in the adult population? 
¢ Does placing a case manager in the ER during peak daytime hours for education of members 


increase overall utilization of primary care services for the adult population? 
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The study questions were clearly stated and should assist in answering the questions raised in 
the topic discussion. The key indicators that the study focused on included: the rates of nurse- 
advice line calls per 1000 members; the rate of ER utilization per 1000 members; and use of a 
PCP or urgent care center as related to the ER visits during 2005. These indicators were 
thoroughly defined in the narrative and included specific measurement guidelines and data 
collection details. This included quarterly claims data queries, identification of the study 


population to ensure proper utilization identification, and quarterly call center data review. 


The study population identified in the narrative documentation was the MC+ MCO members 
seeking care at the Truman Medical Center ER with a non-emergent diagnosis, who appear 
during the 4-6 hours each day that the case manager is present. MC+ MCO members are MC+ 
Medicaid Managed Care recipients in the nine county Western MC+ Managed Care Region. No 


actual sampling was conducted. 


The documentation includes a data analysis collection plan that is appropriately detailed. Data 
will be collected on each member seen in the ER including demographics, reason for visit, 
education provided, barriers identified, and interventions completed. Members were to be 
followed to document post intervention compliance with the agreed upon treatment plan. Data 
analysis was to be performed using control charting, measurements of pre and post-intervention 
effectiveness, assessment of study variables, and ER statistics. Data sources were defined and 
specific. Controls for validity were not clearly identified. The study plan called for reporting to 
the MC+ MCO Internal Utilization Management Committee and Medical Oversight Committee 
on a regular basis. Periodically the Consumer Advisory Committee will receive updates on the 
project as well. The MC+ MCO was also working with a statistician from the University of 
Missouri — Kansas City to ensure that all data was collected, analyzed, and utilized in the most 


efficient and productive manner. 


The main intervention planned was the placement of the case manager in the Truman Medical 
Center emergency room. The details of the activities required were detailed in the narrative. 
This appeared to be a reasonable and creative intervention to impact a problem that is reflected 


in comments from all MC+ MCOs. 
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Data analysis was not completed at the time of submission, but a detailed analysis plan was 
included. This included refining the member population to be followed, particularly for the 
entire post-intervention period. The numbers were affected by a change in Medicaid eligibility 
which occurred in the second half of 2005. The data analysis plan is detailed and comprehensive 
and promised to produce significant results. The initial information indicated that the project 
had a positive impact on MC+ Member health care services. The use of the statistician to assist 
with data analysis added confidence to the validity of the results yet to be published. This 
project was well-constructed and promised to produce significance results that could positively 
influence future methods of impacting how MC+ Members are educated about obtaining 


appropriate and effective health care services. 


STRENGTHS 
|. Excellent study topics and well-defined study questions. 


2. Each Performance Improvement Project was well-developed and included a thorough and 
well-written narrative. 


3. The focus of each project was clearly on improving the health, services, and processes for 
members receiving services. The projects also had the goal of becoming a part of the 
regular performance of the MC+ MCO. 


4. The use of a professional statistician to ensure valid and reliable results indicated the 
investment the MC+ MCO was making in the project underway. 


AREAS OF IMPROVEMENT 
|. Continue development of performance improvement projects that include justification and 


planning, inclusive narratives, and projected outcomes. 


2. Utilize available research and literature reviews to support the projects’ hypotheses and the 
basis for conducting studies. 


RECOMMENDATIONS 
|. Continue the work the MCO is doing with the statistician to perfect PIP methodology. 


2. Encourage staff to continue with these projects and with producing written narratives that 
were being considered for publication. 
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10.2 Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Children’s Mercy Family Health Partners. Children’s Mercy Family 
Health Partners submitted the requested documents on December 9, 2005. The EQRO 
reviewed documentation between December 9, 2005 and February 27, 2005. On-site review 
time was used to conduct follow-up questions and provide feedback and recommendations 


regarding the performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Baseline Assessment Tool (BAT) submitted by Children’s Mercy Family Health Partners for the 
HEDIS 2004 data reporting year 

e  Qualis Health's NCQA HEDIS Compliance Audit Report for HEDIS 2005 

e Children’s Mercy Family Health Partners’ information systems (IS) Policies and Procedures pertaining 
to HEDIS 2005 rate calculation 

e Children’s Mercy Family Health Partners’ information services (IS) policies on disaster recovery 

e Children’s Mercy Family Health Partners’ HEDIS 2005 implementation work plan and HEDIS 
committee agendas for 2005 

e Children’s Mercy Family Health Partners’ HEDIS 2005 Training Manual for the medical record review 
process 

e Documentation, data files and source code of the in-house application for immunization rate 
calculation 

e System edits for the claims management system 


The following are the data files submitted by Children’s Mercy Family Health Partners for review 
by the EQRO: 


e ADVDenom&Num.txt 
e ADVEnrollment.txt 

e ClSDenom&Num.txt 
e ClSEnrollment.txt 

e = ClSHybrid.txt 

e WCIDenom&Num.txt 
e WClEnrollment.txt 


Interviews 

The EQRO conducted on-site interviews with Janet Benson, Johanna Groves, and Jenny Hainey 
at the Children’s Mercy Family Health Partners in Kansas City on Tuesday, March 7, 2005. This 
group was responsible for calculating the HEDIS performance measures. The objective of the 
visit was to verify the data, methods and processes behind the calculation of the three HEDIS 
2005 performance measures. 
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FINDINGS 

Children’s Mercy Family Health Partners used the Administrative Method for calculation of 
Well-Child Visits and Annual Dental Visit measures. The Hybrid Method was used for the 
Childhood Immunization Status measure. MCO to MCO comparisons of the rates of Childhood 
Immunization Status Combination #2, Well-Child Visit, and Annual Dental Visit measures were 
conducted using two-tailed z-tests. For comparisons that were statistically significant at the 95% 
confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), and the 


significance levels (p < .05) were reported. 


The rate for the HEDIS 2005 Childhood Immunization Status, Combination #2 reported to the 
SMA and the State Public Health Agency (SPHA) by Children’s Mercy Family Health Partners 
was 57.47%. This was significantly higher than the statewide rate for MC+ MCOs (28.17%; z = 
12; 95% Cl: 46.37%, 68.57%; p > .95). 


The rate for the HEDIS 2005 Well-Child Visit measure reported to the SMA and the State 
Public Health Agency (SPHA) by Children’s Mercy Family Health Partners was 23.91%. 

This was significantly lower than the statewide rate for MC+ MCOs (38.42%; z = -1.14; 95% Cl: 
11.38%, 36.44%; p < .00I). The rate for Children’s Mercy Family Health Partners for the 2005 
HEDIS Annual Dental Visit measure was 39.10%; significantly higher than the statewide rate for 
MC+ MCOs (29.76%, z = 1.58; 95% Cl: 34.82%, 43.38%; p > .95). 


The following sections summarize the findings of the process for validating each of the 

performance measures in accordance with the Validating Performance Measures Protocol. The 
findings from all review activities are presented according to the EQRO validation activity, with 
the findings for each measure discussed within the activities as appropriate. Please refer to the 


tables in the main report for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

The information systems management policies and procedures for rate calculation were 
evaluated consistent with the Validating Performance Measures Protocol. This included both 
manual and automatic processes of information collection, storing, analyzing and reporting. The 


EQRO was provided with a demonstration of the in-house application developed for calculation 
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of the immunization measure, which used an MS Access data repository for retrieval and 


analysis. 


For all three measures, Children’s Mercy Family Health Partners was found to meet all criteria 
for producing complete and accurate data (see Attachment V: Data Integration and Control 
Findings). There were no biases or errors found in the manner in which they transferred data 
into the repository used for calculating the HEDIS 2005 measures. Children’s Mercy Family 
Health Partners used an external vendor application module for rate calculation. The MC400, a 
product of OAO HealthCare Solutions, Inc was part of the claims management system. The 
module was NCQA-certified for the 2002 HEDIS rate calculation, but the vendor had not 
sought certification since that HEDIS year. The EQRO was provided with a demonstration of 
the MC400, along with the data flow and integration mechanisms for external databases for 


these measures. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures were adequate, as Children’s Mercy 
Family Health Partners had worked diligently within the last year to significantly improve 
documentation of their processes. (See Attachment VII: Data and Processes Used to Calculate 
and Report Performance Measures). Children’s Mercy Family Health Partners met nearly all 
criteria applicable for all three measures. The two criteria that were not met involved the use 
of statistical significance testing to document changes in performance over time and then using 
this information to support claims of improvement or stability in performance over time. 
Although many auditors conduct statistical tests to examine various aspects of data validity, such 
tests should be used by Children’s Mercy Family Health Partners to assess the significance of 
change related to quality improvement activities and operational changes. Children’s Mercy 
Family Health Partners indicated to the EQRO that they intend to utilize statistical significance 


testing prior to next year’s audit. 


Processes Used to Produce Denominators 

Children’s Mercy Family Health Partners met all criteria for the processes employed to produce 
the denominators of all three performance measures (see Attachment X: Denominator 
Validation Findings). This involved the selection of eligible members for the services being 
measured. For the Childhood Immunization Status, Combination #2 measure, a sample of 395 


members were reported and validated. In regards to the Well-Child Visits measure, a total of 
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|,677 eligible members were reported and |,673 eligible members were validated. For the 
denominator of the Annual Dental Visit measure, 24,342 eligible members were reported and 
24,334 were validated. Age ranges, dates of enrollment, medical events, and continuous 
enrollment were programmed to include only those members who met HEDIS 2005 criteria. 
Denominators in the final data files were consistent with those reported on the DST for the 
Childhood Immunization Status, Combination #2 measure. All members were unique and the 
dates of birth ranges were valid. The denominators reported for the Annual Dental Visit 
measure were over-reported by 8 members. 7 of the 8 members were excluded from the 
denominator by the EQRO because they were not enrolled on the anchor date of December 
31, 2004. The eighth member was excluded due to a gap of enrollment greater than 45 days 
during the measurement year. The denominators reported for the Well-Child Visits measure 
were over-reported by 4. The EQRO excluded those 4 members because they were not 
enrolled in the Children’s Mercy Family Health Partners health plan on the anchor date of their 


|5-month birthday. 


Processes Used to Produce Numerators 
All three measures included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2005 criteria 


(see Attachment XIll: Numerator Validation Findings). 


For the HEDIS 2005 Childhood Immunization Status, Combination #2 measure, Children’s 
Mercy Family Health Partners appropriately included administrative events from the State Public 
Health Immunization Registry (MOHSAIC). The EQRO’s review of the administrative hits 
validated 109 of the 395 records provided. Thirty (30) of 30 medical records requested were 
received; 29 records resulted in validated hybrid hits. As a result, the medical record review 
validated | 17 of the 121 hybrid hits reported. Based on the number of hits validated by the 
EQRO, the rate calculated was 57.21%. The total estimated bias for the Childhood 


Immunization Status, Combination # 2 measure was a .26% overestimate of the rate. 


Children’s Mercy Family Health Partners used the Administrative Method to calculate HEDIS 
2005 Well-Child Visits measure. Review of the administrative hits validated 400 of the 401 hits 
found by the MCO. The rate calculated by the EQRO was 23.85%, with a bias of 0.06%, an 


overestimate by the MCO in the reporting of the measure. 
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The HEDIS 2005 Annual Dental Visit measure was the third measure validated. Review of the 
administrative hits validated 9,516 hits in the files provided for review by the EQRO. Although 
the MCO reported 9,517 hits, they only provided the EQRO with files containing 9,516 records. 
The dates of birth, dates of service, and dates of enrollment were in the correct range and valid. 


The rate calculated by the EQRO was 39.09%, with no bias in the reporting of the measure. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Childhood Immunization Status, Combination #2 measure. 
Attachment XIl; Impact of Medical Record Review Findings and Attachment XV: Sampling 
Validation Findings were completed for each of these measures. Children’s Mercy Family Health 


Partners was compliant with all specifications for sampling processes. 


Submission of Measures to the State 

Children’s Mercy Family Health Partners submitted the DST for each of the three measures 
validated to the SPHA (the Missouri Department of Health and Senior Services) in accordance 
with the Code of State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance 


Organizations) and the SMA Quality Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 

The following tables summarize the estimated bias in reporting each of the measures and the 
final validation findings. Table 78 shows a small overestimate (inside the 95% confidence 
interval) for the Childhood Immunization Status, Combination #2 and Well- Child Visits 


measures, and no bias observed in the calculation of the Annual Dental Visits measure. 


Table 78 - Estimate of Bias in Reporting of HEDIS 2005 Measures 


Estimate of Total 
Measure Bias Direction of Estimate 


Childhood Immunization Status, Combination #2 0.26% overestimate 


Well-Child Visits in first 15 months of Life 0.06% overestimate 
Annual Dental Visit None 





Final Audit Rating 

The Final Audit Rating for each of the performance measures was based on the findings from all 
data sources that were summarized in the Final Performance Measure Validation Worksheet. 
Table 79 shows the final audit findings for each measure. Both the Childhood Immunization 


Status and Well-Child Visits measures were Substantially Compliant, as there was no significant 
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bias associated with the overestimated rates. The Annual Dental Visit was Fully Compliant with 


State specifications. 


Table 79 - Final Audit Rating for HEDIS 2005 Performance Measures 
Measure Final Audit Rating 


Childhood Immunization Status, Combination #2 Substantially Compliant 

Well-Child Visits in first 15 months of Life Substantially Compliant 

Annual Dental Visit Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by the MCO. Not Valid = Measure deviated from State specifications such 
that the reported rate was significantly biased. This designation is also assigned to measures for which no rate was 
reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Children’s Mercy Family Health Partners effectively integrates the MOHSAIC data into 
administrative rates for calculation of immunization rates. The sample files are sent to the 
SPHA for identifying members with an immunization history. The SHPA returns a file with 
relevant immunization information for Children’s Mercy Family Health Partners’ members. 
MOHSAIC data comprises a majority of hits within the administrative hits. 


2. Rates for the HEDIS 2005 Childhood Immunization Status, Combination #2 and Annual 
Dental Visits measures were significantly higher than the average for all MC+ MCOs. 


3. There was a well-designed HEDIS reporting module of the MC400 system. This module 
was certified by NCQA in 2002, but was not certified for 2004. Audit checks were 
conducted on this system by NCQA-certified auditors on an annual basis to confirm the 
system’s output. The system generates appropriate production logs to track errors. 


4. Data from external sources, such as the pharmacy vendor AdvancePCS, and data from 
medical record review was efficiently integrated. The format of data mapped in the 
integration process is automatically validated and also manually checked. 


5. There were effective validity checks through automated system edits. The claims system 
had internal edit checks for each transaction and an error handling mechanism that validated 
the process. 


6. Children’s Mercy Family Health Partners had a separate application for combining the 
immunization data from medical record reviews and administrative searches. An MS Access 
database acted as a data repository for the application. This integrated the data from the 
claims system and effectively combined it with external data to produce the final 
immunization rates. This application was well-documented and produced accurate results. 
The data fields were clearly defined and business rules were well-documented. 


7. Children’s Mercy Family Health Partners had significantly improved its documentation of 
processes and organization of data. 


8. Two members of the Children’s Mercy Family Health Partners HEDIS team attended HEDIS 
training in October 2005. 
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AREAS FOR IMPROVEMENT 


|. Data analysis should incorporate tests of statistical significance to assess whether the 
observed changes in rates are related to a specific intervention. 


2. The HEDIS 2005 Well-Child Visits in the First |5 months of Life measure rate was 
significantly lower than the average for all MC+ MCOs. 


RECOMMENDATIONS 
|. Conduct and document statistical comparisons on rates from year to year. 


2. Children’s Mercy Family Health Partners should consider doing Administration/Hybrid 
optional measures as opposed to Hybrid, to find more hits. 
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10.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical fields? 


For the Medical claim type, there were | 17,554 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 

The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 

The Outpatient First Date of Service field was 100.00% complete, accurate and valid. 
The Outpatient Last Date of Service field was 100.00% complete and accurate, and valid. 


The Outpatient Units of Service field was |00.00% complete accurate and valid. 


~~ oe i 6) 


The Outpatient Procedure Code field was 100.00% complete and accurate, with 99.77% 
containing valid values. The following are the codes found in the 270 invalid fields. 





Frequency Code 
138 99502 

37 9960 | 

93 B4035 

| E2603 

| K0638 


7. The Outpatient Place of Service field was 100.00% complete, accurate and valid. 
8. The first Diagnosis Code field was |00.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the 
Health Plan Record Layout Manual, they fell well below the 100% threshold established by 
the SMA for this validation. The second Diagnosis Code field was 44.56% complete, 
accurate and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). 
The third Diagnosis Code field was 13.84% complete, accurate and valid (incomplete, 
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inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and 
invalid). The fourth Diagnosis Code field was 6.91% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code 
field was 0.00 % complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). 


For the Dental claim type, there were 16,061 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. All of the fields examined were 100.00% complete, 


accurate and valid. 


For the Home Health claim type, there were 36 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. All fields examined were |00.00% complete, 
accurate, and valid except for the second through fifth Diagnosis Code fields. The second, third, 
fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for completeness, 
accuracy, and validity established by the SMA (86.11%, I 1.11%, 11.11%, and 0.00%, respectively). 


The remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Inpatient claim type, there were 15,299 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


|. The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Recipient ID field was 100.00% complete, accurate and valid. 

3. The Admission Type field was 100.00% complete, accurate and valid. 
4 


The Admission Date field was 100.00% complete and accurate; and 96.24% valid. There 
were 576 invalid dates ranging from 12/01/2005 — 12/31/2005. 


The Discharge Date field was 100.00% complete and accurate; and 96.50% valid. There 
were 536 invalid dates ranging from 04/01/2005 — 05/23/2005. 


— 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


0 1 ND 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold 
for completeness, accuracy, and validity established by the SMA (92.01, 70.46%, 53.70%, and 
15.71%, respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


10. The First Date of Service field was 100.00% complete and accurate, and 96.24% valid. There 
were 576 invalid dates of service ranging from 12/01/2005 — 12/31/2005. 


ll. The Last Date of Service field was 100.00% complete and accurate, and 96.50% valid. There 
were 536 invalid dates of service ranging from 04/01/2005 — 05/23/2005. 
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|2. The Revenue Code field was 99.98% complete, accurate, and valid. There were 3 fields left 
blank. 


|3. The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were 94,65! encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. 


|. The Outpatient Hospital Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete and accurate, and valid. 

The Last Date of Service field was 100.00% complete and accurate, and valid. 


The Units of Service field was 100.00% complete, accurate and valid. 


> we YY 


The Outpatient Procedure Code field was 100.00% complete and accurate, and 99.99% 


valid. This field requires five alphanumeric characters. There were |2 invalid fields of 
“99999”. 


7. The Outpatient Hospital Revenue Code field was 100.00% complete, and 100.00% (with 
rounding) accurate and valid. The two invalid fields contained | value of “30” and | value of 
“70”. This field requires a three-digit code ranging 100-999. 


8. The first Diagnosis Code field was |00.00% complete, accurate and valid. 


9. Although the second through fifth Diagnosis Code fields are optional according to the 
Health Plan Record Layout Manual, they fell well below the 100% threshold established by 
the SMA for this validation. The second Diagnosis Code field was 55.65% complete, 
accurate and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). 
The third Diagnosis Code field was 20.88% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and 
invalid). The fourth Diagnosis Code field was 9.49% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code 
field was 1.02% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). 


For the Pharmacy claim type, there were 92,618 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. All fields examined were |00.00% complete, accurate and valid 


data for all fields examined. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Family Health Partners, an error analysis of the invalid entries 
was conducted for fields which were lower than the |00.00% threshold specified by the SMA. 
Dental, Home Health, and Pharmacy claim type fields examined were |00.00% complete, 
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accurate, and valid (see previous findings). The Admission Date, Discharge Date, and Outpatient 
Procedure Code fields in the Medical claim type contained invalid procedure codes. For the 
Hospital Outpatient claim type, there were invalid codes for the Outpatient Procedure and 


Revenue Code fields. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, the rates of Inpatient, 
Medical, Pharmacy, and Home Health claim types were consistent with the average for all MC+ 
MCOs, while the rates for Dental and Outpatient Hospital claim types were significantly higher 
than the average for all MC+MCOs. This suggests that the data are complete and that there is 
better utilization of dental services and high rates of access to preventive and acute care among 


Family Health Partners members. 


To What Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 
types for the period of January |, 2005 through March 31, 2005 for medical record review. Of 
the 117,554 Medical encounter claim types in the SMA extract file for January |, 2005 through 
March 31, 2005, 100 encounters were randomly selected. Providers were requested to submit 
medical records for review. There were 98 medical records (98.0%) submitted for review. 
Encounters for which no documentation was submitted were unable to be validated. The match 
rate for procedures was 50.0%, with a fault rate of 50.0%. The match rate for diagnoses was 


98.0%, with a fault rate of 2.0%. 


What Types of Errors Were Noted? 

An error analysis of the errors found in the medical record for procedure and diagnosis codes 
was conducted. For the diagnosis code in the medical record, the reasons for diagnosis codes 
not matching the SMA extract file was missing information (n = 2) with no incorrect 


information. The diagnosis code listed did not match the descriptive information in the record. 
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For the procedure code in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = 31) and not enough information to code (n=19). Examples of missing 
information included no code, codes listed that were not supported, or codes that did not 


match the procedure description. 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to 
the SMA encounter claim extract file due to the absence of an internal control number or 
crosswalk to compare the MC+ MCO files to the SMA extract files. The plan was to compare 
MC+ MCO paid and unpaid claims to the SMA encounter claim extract file. MC+ MCOs were 
requested to submit data, as specified by the EQRO (see Appendix 6), for the MC+ Managed 


Care Members represented in the encounter claim sample selected for validation. 


For purposes of the EQRO, Family Health Partners was able to submit files in the requested 
format, but the ICN codes submitted were not the same number of characters of the ICNs in 


the SMA extract files. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While the MC+ MCO did submit the data in the requested format, there are a number of ways 
to improve the data quality by improving the database system. One variable that is not 
currently represented is that of a unique line number. To match up specific lines of data (each 
service provided within an encounter), this requires a unique number for each service provided 


for each member. 
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STRENGTHS 


Encounter data was submitted to the EQRO in the requested format which allowed 
encounter validation for all claim types. 


The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


The critical fields evaluated for the Dental, Home Health and Pharmacy claim types were 
|00.00% complete, accurate, and valid. 


The match rate between the medical record and SMA encounter claims data was statistically 
higher than the average for all MC+ MCOs for the diagnosis code. 


AREAS FOR IMPROVEMENT 


The Outpatient Procedure Code field for the Medical claim type contained invalid codes. 


2. The Revenue Code for the Inpatient claim type contained 3 blank fields. 

3. The Outpatient Procedure and Revenue Code fields in the Outpatient Hospital claim type 
contained invalid codes. 

4. The match rate between the medical record and SMA encounter claims data was 
comparable to the average for all MC+ MCOs for the procedure code. 

RECOMMENDATIONS 

|. Examine and revise as needed internal system edits for invalid procedure codes in the 
NSF/CMS 1500 file layout for the Outpatient Procedure Code and run validity checks after 
the programming of new edits. 

2. Ensure that the Outpatient Procedure and Revenue Code fields are complete and valid for 
the Outpatient Hospital claim types, and institute error checks to identify invalid data. 

3. For audit purposes, submit extract files for encounter data in the requested file layouts with 


the requested ICN codes that are identical to the SMA extract file. 
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10.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract documents with the staff of the Division of Medical Services. On-site review 
time was used to conduct interviews with those who oversee the daily practices of the MCO. 
This ensures that documentation is developed and practices occur within the scope of the 


contract and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The following documents pertaining to Children’s Mercy Family Health Partners were reviewed 
prior to and at the on-site visit: 


The Division of Medical Services supplied: 
» State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

=» Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection process of actions filed in the first quarter of 2005. 

# 2004 Annual Quality Improvement Program Evaluation 


Additional documentation made available by Children’s Mercy Family Health Partners included: 
# 2005 and 2006 Marketing Plan 
= Children’s Mercy Family Health Partners’ Organizational Chart 
= Corporate Dashboard Indicators CY 2002-2006 
# 2005 Welcome Calls summary by Quarter 
= New Member Welcome Call Script 
= Connection — Member Newsletter 
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area A division of Behavioral Health Concepts, Inc. 353 


Missouri MC+ Managed Care External Quality Review Section 10 
Report of Findings — 2005 Children’s Mercy Family Health Partners 


= Are You Pregnant and Need Health Coverage brochure (2 languages) 

= Do Your Children Need Health Coverage brochure (2 languages) 

#» Use Your Time Wisely brochure (2 languages) 

= Resource Paper: Kansas City Children’s Asthma Management Program: KC CAMP 
Family Health Partners (funded by the Center for Health Care Strategies) 

= Health Management Quarterly Subcommittee Meeting Minutes — January, April, July, 
October 2005 

= Community Relations 2005 Annual Report 

# Children’s Mercy Family Health Partners Initial Physician Utilization and Cost 
Comparison Profile Report 


Interviews 

Interviews were conducted with the following group: 

Plan Administration 
Robert Finuf — Chief Executive Officer, Plan Administrator 
Ma’ata Touslee — Director of Health Services 
Jenny Hainey — Manager, Quality Management 


Linda Steinke — Director of Operations 
Dr. Rubin — Medical Director 


Mental Health 
Brian Baker — CommCare 
Lynn Durbin — CommCare 
Eric Schmidt — CommCare 
Ma’ata Touslee - CMFHP 
Jenny Hainey — CMFHP 
Linda Steinke — CMFHP 


FINDINGS 

Enrollee Rights and Protections 

The staff at Children’s Mercy Family Health Partners (CMFHP) continued to exhibit a strong 
commitment to ensuring that member rights were protected. The MCO utilized interpreter 
services, pre-translated written materials and a variety of methods for those members who 
spoke a language other than English. The MCO provided alternatives to members who may 
have reading, vision, or hearing problems that enabled them to obtain required information 
about the health plan or the services they can expect to receive. Member Services staff set up 
alternatives for individuals with any barrier to obtaining services and worked diligently to ensure 


that they received any necessary assistance. 
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During 2005, CMFHP developed a tracking system to guarantee that all required materials and 
policy are reviewed on an annual basis, as required, and are submitted to the SMA in a timely 
manner. This information is reviewed on a monthly basis and is stored in a locally maintained 
Access database. A quality committee reviewed the database information quarterly to ensure 
that all updates occurred timely. Member education and marketing materials were all submitted 


and approved during 2005. 


CMFHP worked with an external contractor to develop applications of the ManagedCare.com 
software for their health information system. The company, using an internal utilization 
management committee, initially looked at all parts of the CMFHP system and narrowed the 
initial focus to ten areas. One of the preliminary projects involved completing provider profiling 
information in an effort to create a “report card” for providers. The MCO used the first part of 
2005 to input and validate data. Profiling will be completed two times per year on all providers. 
The MCO also plans to use this software to create pharmacy profiles for members who have 


had difficulty managing their medication. 


During 2005 the CMFHP Member Advisory Committee was established, and met four times 
during the year. The MCO exhibited its strong commitment to the advisory committee 
members by sending reminders and paying for transportation, thus eliminating this potential 
barrier to attendance. CMFHP raised issues with the committee and utilized their 
recommendations whenever possible. Examples included changing the format of the Member 
Handbook to include the Provider Directory in one document, changing the Welcome Call 
script to enhance understanding, and redesigning the appearance of brochures to be more 
appealing to the intended audience. CMFHP, upon committee recommendation, revamped their 
external Website to make it more user friendly. Reminders on enrollment, EPSDT, and other 


“hot topics” were created and sent out to members on a regular basis. 


During, 2005 CMFHP made attempts to inform, engage, and reach out to the Latino community 
by employing an individual of Latin dissent. As a result, the MCO Welcome Call was amended 
and now includes a Spanish version. All written information regularly used by members is now 
automatically produced in Spanish. In addition, a Spanish language component became part of 


the Nurse Advice Line to ensure immediate availability to Spanish speaking members. 
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Ratings for Compliance with Enrollee Rights and Protections (100%) reflected policy and 
procedures that were submitted to and approved by the SMA. All written information has been 
submitted for approval. All practice observed, as well as additional documentation viewed while 


on-site, indicated that the MCO is fully compliant in this area. 


Table 80 - Subpart C: Enrollee Rights and Protections Yearly Comparison 


Federal Regulation 





438.100(a) Enrollee Rights: General Rule 1 2 
438.10(b) Enrollee Rights: Information Requirements 2 2 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 
438.10(c)(4,5) Language and Format: Interpreter Services 2 2 
438.10(d)(1)(i) Information Requirements: Format/Easily 

2 2 
Understood 
438.10(d)(1)(ji)and (2) Information Requirements: Format Visually 5 5 
Impaired, and Limited Reading Proficiency 
438.10(f) Information for All Enrollees: Free Choice, etc. 2 2 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 5 5 
Plans 
438.10(i) Special Rules: Liability for Payment/Cost Sharing 2 2 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee Communications 2 2 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 1 2 
438.100(b)(3) Right to Services 1 2 
438.100(d) Compliance with Other Federal/State Laws Ps 2 
Number Met 10 13 
Number Partially Met 3 0 
Number Not Met 0 0 
Rate Met 76.9% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Behavioral Health 

CMFHP continued to work with CommCare for the provision of behavioral health services for 
members. This relationship continued as a strong partnership with a focus for providing 
effective services to members. CommCare was questioned about their policy regarding 
transferring members receiving services from out-of-network providers as a follow-up to the 
2004 review. The BHO stated that they did explore changing the policy. The decision was 
made to continue with a four-week window of continued services with the out-of-network 


provider. This would allow the provider to become a member of their network, or to 
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transition the member to a new provider. The BHO recognized that this may restrict members 
in some situations, but reported that they had very few member complaints. CommCare stated 
that there was no evidence that any harm had come to members required to make a change. 
BHO staff provided assurances that when it appeared a clinical reason existed to maintain a 
therapeutic relationship with an out-of-network provider, all relevant factors would be 
considered. A decision would be made in the member’s best interest. CommCare reported 
that they have recruited a number of new providers through these transitional cases, and always 


encourage providers to become a network member. 


CommCare attempted to decrease the rate of readmissions following hospitalization and 
reported results as a follow-up to the 2004 review. The BHO engaged physicians to assist them 
in identifying issues that were triggers to readmission. CMFHP and CommCare staff, hospitals, 
and network providers were included in the review. This group identified that a key issue 
impacting readmissions, involved members who received inpatient treatment and had limited 
family support upon discharge. As a result, the BHO developed a system to provide in-home 
therapy and intensive case management for members at risk. CommCare agreed to absorb any 
cost for additional encounters not reimbursed by the SMA, such as late night telephone 
interventions. The BHO contracted with three specific providers to perform the intensive case 
management component of this service system. The result of this improved approach has been 
thirty-one fewer hospital readmissions July through December 2005. CMFHP and CommCare 
are assessing the clinical improvement achieved by these members. A committee is tracking the 
progress of the initiative. Another positive effect of the attention given to prevention of 
readmissions has been an improved rate of post-hospitalization follow-up for members in seven 
and thirty days after discharge. Emphasis was placed on discharge planning through work with 
hospital-based case managers and discharge planners. The goal was to initiate discharge planning 


within the first 24 hours of hospitalization. 


A Psychiatric drug committee was formed during 2005, which included staff from the MCO, 
CareMark, a physician from Children’s Mercy Hospital, and the medical director from 
CommCare. The committee met quarterly to review data pertaining to ADHD diagnosis, 
antipsychotic medications, and antidepressant drugs and discussed additional clinical findings 
available in the literature. The MCO did follow-up with physicians regarding prescription habits 
and published clinical findings that are applicable. Some formulary changes occurred to place 


edits on specific medications. All Community Mental Health Centers (CMHCs) were given 
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updated information about these changes from the physician committee members. The 
committee was in the process of developing training modules on polypharmacy issues at the 
time of the review. Education was planned for provider offices during the coming year and for 


providing a contact for consultation. 


Quality Assessment and Performance Improvement 


Access Standards 

CMFHP continued to have a strong provider network through the MC+ Region. The MCO has 
worked one-on-one with providers, including specialists who agreed to become panel members. 
The MCO recognized a continued need for neurosurgeons and orthopedic surgeons. CMFHP 
recruited several specialists who agreed to be in the network, but requested to remain silent 
and not be published in the Provider Manual. These providers saw members when contacted 
directly by MCO staff. The MCO also engaged Truman Medical Center in this process, to 
ensure that members were triaged and received a referral and provider access quickly. CMFHP 
continued to monitor their PCP availability and continued recruitment to ensure that adequate 


open panels were available. 


Prior authorization was an area of concern for the MCO. A committee of nurses was 
developed to review prior authorizations. They found that a number of procedures and 
medications existed that were always approved, and others that indicated a need for case 
management. The two medical directors also recognized that several types of denied claims, 
which were regularly overturned, were a recurring problem. They created a number of 
scenarios for the nurses to follow to allow for automatic approval. This measure offset any 
delay that might have occurred waiting for medical director review. The health information 
system had been programmed to automatically review certain procedures and medications. 
Analysis indicated that several of these procedure requests were routinely approved after the 
initial system generated denial. Therefore, the system was adjusted to allow for approval of 


certain procedures without prior authorization or medical director approval. 


CMFHP worked with their subcontractor, Bridgeport Dental, to improve access to orthodontic 
services. Bridgeport used restrictive criteria for the authorization of braces. The MCO 
provided training to ensure the same criteria was applied as that used by the SMA for approval 


of braces. This decreased the number of appeals filed and overturned. 
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The MCO continued to use member surveys and on-site reviews to monitor access standards. 
When deficiencies were identified they were dealt with in writing. Direct provider contact 


occurred where required. Re-audits occurred to ensure that improvement was sustained. 


Ratings for compliance with Access Standards (100%) reflected completion of all required 
written policies and procedures. Observations and interviews that occurred during the on-site 
review provided additional evidence that MCO practices and operations appear to be compliant 


with the MC+ Managed Care Contract and federal regulations. 


Table 81 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 


CMCMFHP 


Federal Regulation 





438.206(b)(1)(i-v) Availability of Services: Provider Network 
438.206 (b) (2) Access to Well Woman Care: Direct Access 


438.206(b)(3) Second Opinions 


438.206(b)(4) Out of Network Services: Adequate and Timely 
Coverage 


438.206(b)(5) Out of Network Services: Cost Sharing 
438.206(c)(1)(i-vi) Timely Access 

438.206(c)(2) Provider Services: Cultural Competency 
438.208(b) Care Coordination: Primary Care 

438.208(c)(1) Care Coordination: |Identification 
438.208(c)(2) Care Coordination: Assessment 

438.208(c)(3) Care Coordination: Treatment Plans 
438.208(c)(4) Care Coordination: Direct Access to Specialists 
438.210(b) Authorization of Services 

438.210(c) Notice of Adverse Action 


Re NY N /N 


438.210(d) Timeframes for Decisions, Expedited Authorizations 


438.210(e) Compensation of Utilization Management Activities 


NO NY NY NY NIN NY NY NY NY NN NY NY N N WN 


PN NN FF FP NY NY N fF NN N WN 


438.114 Emergency and Post-Stabilization Services 
Number Met 12 
Number Partially Met 5 
Number Not Met 0 0 
Rate Met 70.6% 100.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 
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Structures and Operation Standards 

CMFHP members have open access to specialists, with no referral from the PCP required. In 
some cases members receive assistance with referrals from MCO case managers. When a 
member had a specific problem, and care coordination was needed between clinicians, this was 
provided by the appropriate case manager. The MCO initiated a formal means of facilitating 
communication between PCPs and BHO providers, with member approval, to ensure that 


pertinent information was shared. 


CMFHP formed a committee during the past year to discuss how the best methodology for 
making information about advance directives available to members. The goal was to have this 
information available at PCP offices. Education and materials were provided to PCPs on this 
topic. Two areas that remained problematic were accurate completion of all required 
documentation and proper recording in medical records. The MCO continued to work with 


PCP offices to improve these areas. 


CMFHP participated in meetings during 2005 where discussions occurred regarding methods of 
reducing the number of high risk pregnancies in the MC+ Medicaid Managed Care population. 
The MCOs shared forms used, methods of identification of pregnant members, and methods of 
collecting information from members that identify potential high-risk situations. A pregnancy 
notification form was adopted. CMFHP recognized an improvement in the number of health 
assessments returned by members. This group’s activities were reported to the SMA. CMFPH 
also reported joining a national group of Medicaid only organizations that share ideas and 
information on member services. The MCO hoped to be able to offer an incentive to pregnant 


members for attending all physician visits during their pre and post delivery time periods. 


The ratings for compliance with Structure and Operation Standards (100%) reflected complete 
policy and procedural requirements. The MCO appears to be compliant with all policy and 


practice in this area that meets SMA contract compliance and federal regulations. 
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Table 82 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


-CMCMFHP 


Federal Regulation 





438.214(a,b) Provider Selection: Credentialing/Recredentialing 1 2 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 2 2 
438.214(d) Provider Selection: Excluded Providers 0 2 
438.214(e) Provider Selection: State Requirements 1 2 
438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 5 5 
limitations 

438.56(c) Disenrollment Requested by the Enrollee 2 2 
438.56(d) Disenrollment: Procedures 2 2 
438.56(e) Disenrollment: Timeframes 2 2 
438.228 Grievance System 2 2 
438.230(a,b) Subcontractual Relationships and Delegation 1 2 
Number Met 6 10 
Number Partially Met 3 0 
Number Not Met 1 0 


Rate Met 60.0% 100.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Measurement and Improvement 

CMFHP continued to be an active member of the Kansas City Quality Improvement 
Consortium (KCQIC) and utilized the practice guidelines developed and supported by that 
group. The local guidelines that were used by the MCO continued to meet or exceed nationally 
accepted standards. The KCQIC was in the process of developing guidelines on obesity 
treatment. CMFHP will implement these guidelines upon their completion. The MCO 


continued to utilize Milliman and Roberson guidelines for utilization management. 


CMFHP did submit two Performance Improvement Projects (PIPs) for validation. Specific details 
of these projects can be found in the appropriate section of the report. It was noted that the 
MCO utilized projects that had been started, and perfected these projects in an effort to create 
improved services to members during the measurement year. These PIPs were well- 


constructed and provided adequate information for validation. 
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The MCO submitted all required information to complete the Validation of Performance 

Measures, as requested. CMFHP continued to operate a health information system within the 
guidelines of that protocol. All encounter data requested was provided in the correct format. 
The details of each of these areas of validation can be reviewed within specific sections of this 


report. 


Ratings for the Measurement and Improvement sections were found to be (100%), which 
reflects that all required policy and practice meets the requirements of the MC+ Medicaid 


Managed Care contract and the federal regulations. 


Table 83 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison 





Federal Regulation peel 
_ 2004 | 2005 

438.236(b)(1-4) Practice Guidelines: Adoption 2 2 
438.236(c) Practice Guidelines: Dissemination 2 2 
438.236(d) Practice Guidelines: Application 2 2 
438.240(a)(1) QAPI: General Rules 2 2 
438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 1 5 
Improvement and PIPs 
438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 2 2 
438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 2 2 
438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare 5 5 
Needs 
438.240(e) QAPI: Program Review by State NA NA 
438.242(a) Health Information Systems 2 2 
438.242(b)(1,2) Health Information Systems: Basic Elements 1 
438.242(b)(3) Health Information Systems: Basic Elements 1 2 
Number Met 8 11 
Number Partially Met 3 0 
Number Not Met 0 0 


Rate Met 72.1% 100.0% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Grievance Systems 

Ratings for compliance with the Grievance Systems regulations (100%) indicate that the MCO 
completed all requirements regarding policy and practice. Four member appeals and two 
member grievances were reviewed. Two of the member appeals were requests to review 
denials for cosmetic reasons. These were reviewed and overturned by the MCO. The third 
appeal concerned a denial for orthodontia, which was reviewed appropriately. The final finding 
was that criteria were not met. The fourth appeal was also upheld and this appeared within 
policy. Of the two member grievances, one was upheld. Although it stemmed from a concern 
over vehicle safety, the vehicle was inspected and found to be safe. The driver was counseled 
about safety and timeliness issues. The other pertained to inappropriate waiting time in a dental 
office and rude staff. The member was given an appointment with another provider. The 
original provider and staff were counseled about member treatment. A letter of apology was 


sent to the member from this medical group. 


Six complaint files were reviewed concerning provider issues. Two were upheld due to lack of 
authorizations. Three were overturned with the receipt of additional information. The 
remaining complaint concerned what appeared to be a request for cosmetic surgery. VVith 
additional information it was learned that this concerned a mass that had changed or enlarged. 
lt was also overturned. All member and provider files were in order and contained required 
and approved notification. All correspondence was sent within policy timeframes. Medical 


directors were appropriately involved to ensure that members obtained the healthcare they 


required. 
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Table 84 - Subpart F: Grievance Systems Yearly Comparison 





Federal Regulation 


438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2 
438.404(a) Grievance System: Notice of Action - Language and 5 5 
Format 
438.404(b) Notice of Action: Content 2 2 
438.404(c) Notice of Action: Timing 2 2 
438.406(a) Handling of Grievances and Appeals: General 5 5 
Requirements 
438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 
438.408(a) Resolution and Notification: Basic Rule 
438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 
438.408(d)(e) Resolution and Notification: Grievance and Appeals - 

2 2 
Format and Content of Notice 
438.408(f) Resolution and Notification: Grievances and Appeals - 5 5 
Requirements for State Fair Hearings 
438.410 Expedited Resolution of Appeals 2 2 
438.414 Information about the Grievance System to Providers and 5 5 
Subcontractors 
438.416 Recordkeeping and Reporting Requirements 2 2 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 18 18 
Number Partially Met 0 0 
Number Not Met 0 0 
Rate Met 100% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 


Summary 

Children’s Mercy Family Health Partners improved in meeting all policy, procedure, and practice 
areas of compliance with both the MC+ Medicaid Managed Care contract requirements and the 
federal regulations. The MCO exhibited a meticulous attention to meeting all the details of the 
regulations, submitting policy and procedural updates in a timely fashion, and utilizing the 2004 
External Quality Review as a guideline for meeting required standards. The staff within CMFHP 
exhibited a commitment to excellence in serving MC+ Medicaid Managed Care members. They 
demonstrated respect and dignity toward members, while meeting their healthcare service 
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needs efficiently and effectively. The MCO went beyond the strict requirements of their 
contract to ensure that members are able to have a voice in the design of their healthcare 
system. The system created at CMFHP is responsive and strives to assist its members in 
overcoming the barriers often encountered in the areas of quality, access and timeliness in 


obtaining healthcare services. 


STRENGTHS 


|. Completion and approval of all policy and procedures required by the MC+ Managed Care 
Contract. 

2. Development of an improved tracking system for all materials and policy that must be 
submitted annually, including monthly review of submission status. 

3. Development and utilization of a Member Advisory Committee that meets quarterly. 
Commitment to assist members’ attendance, including reminders and transportation. 

4. Utilization of members’ suggestions, such as combining the Member Handbook and Provider 
Listing into one document. 

5. Commitment to the Latino community by employing a staff member for the purpose of 
community outreach and implementation of member suggestions. 

6. Creative approach to engaging providers, particularly in hard-to-reach specializations. 

7. Actively engaging new health management programs to benefit members. Working with 
Children’s Mercy Hospital on a childhood obesity outreach program, PFITT. 

8. Treatment of members and providers during the grievance and appeal process. In one 
record the provider sent a letter of apology to the member. 


AREAS FOR IMPROVEMENT 


|. Continue vigilant attention to continuous improvement within the organization and 
attention to improving services to members. 


RECOMMENDATIONS 


|. Continue to develop an organization that can exhibit energy and enthusiasm for its mission. 

2. Continue to actively monitor providers and subcontractors and to develop corrective action 
initiatives when a problem is identified, such as advance directive utilization. 

3. Continue to look for creative methods to use as motivators, such as available incentives, to 
encourage member utilization of MCO resources, particularly for high-risk populations. 


Gee Performance Management Solutions Group 


Group 


tances es A division of Behavioral Health Concepts, Inc. 365 


Missouri MC+ Managed Care External Quality Review Section | | 
Report of Findings — 2005 FirstGuard Health Plan 


~) 1 OS BLO ER! 


FIRSTGUARD HEALTH PLAN 





Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 366 





Missouri MC+ Managed Care External Quality Review Section | | 
Report of Findings — 2005 FirstGuard Health Plan 


(This page intentionally left blank.) 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 367 





Missouri MC+ Managed Care External Quality Review Section || 
Report of Findings — 2005 FirstGuard Health Plan 





7 


11.0— FirstGiiatil 


Thee Peart coved soul a” dee lnieoure.* 


[1.1 Performance Improvement Projects 


The previous sections of the 2005 EQRO report present the purpose and objectives, technical 
methods, procedures for evaluation, and MCO to MCO comparisons of analyses, findings, and 
recommendations for each of the protocols based on the aggregate analysis. This section of the 
EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed 
Care Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


METHODS 


Document Review 


FirstGuard Health Plan supplied the following documentation for review: 


= NCQA Quality Improvement Activity Form: Improving Asthma Medication Management 
= NCQA Quality Improvement Activity Form: Improving Birthweight Outcomes 


Interviews 


Interviews were conducted with the project leaders for each Performance Improvement Project 
(PIP) by the EQRO team on March 10, 2006 during the on-site review, and included the 


following: 


Jean Rumbaugh — COO, Plan Administrator 

Dr. William Pankey — VP, Medical Affairs 

Barbara Maloney — VP, Development and Regulatory Affairs 

Celia Humphreys — Director, Quality Improvement and Credentialing 
Susan Richart — Senior Business Analyst 

Sandra Clausen 
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Interviewees shared information on the validation methods, study design, and findings. Technical 
assistance regarding study design and presentation of findings was provided by the EQRO. The 
following questions were addressed: 


= = —=Who was the project leader? 

= How were the topics identified? 

= How were the study questions determined? 
= =~=What were the findings? 

= = =What interventions were chosen? 

= How effective were these interventions? 


Additional information was provided at the time of the on-site review and was considered in the 


final validation of these Performance Improvement Projects. 


FINDINGS 

The first PIP evaluated was “Improving Asthma Medication Management.” Topic selection was 
clearly defined. The MC+ MCO had worked with the Kansas City Quality Improvement 
Consortium (KC-QIC) through 2002 to develop regionally specific practice guidelines for 
asthma management and treatment. Part of this project included guidelines for asthma 
medication management. The problems associated with uncontrolled asthma were discussed in 
the narrative. National data and a national literature review references were utilized in 
developing this topic. This information, coupled with regional and MC+ MCO data, supported 
the decision to choose this topic as a valid Performance Improvement Project (PIP). The 
information provided also identified the indicators of uncontrolled asthma, as well as new and 
improved asthma medication management strategies. This was incorporated into the study. 
The stated goal of the study was to positively impact the use of preventive services by educating 
providers and members who depend on rescue medication over the use of regular and 
preventive medications. The parameters of the population included were clearly defined as all 
members ages 3-56 with special health care needs, particularly asthma. The research supported 


the age range targeted due to the increased need and diagnosability of this group. 


There is not a clearly specified study question presented. The question inferred was “Is an 
intervention that was based on educational mailing to providers and members, adequate to 
improve asthma medication usage over time?” The information provided did not distinguish a 


measurable or strongly worded study question. 
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The study did include objective and clearly defined indicators. The narrative delineated 
quantifiable measures that included baseline measurements. The indicators included were: 
identification of MC+ MCO members needing asthma management and MC+ MCO asthmatics 
with suboptimal asthma medication management. The population was clearly defined and was 
justified by the information provided. The population included MC+ Members and members of 
the Healthwave program, which is the Kansas Medicaid Managed Care program. How data on 
the two groups would be distinguished was not presented and could lead to confusion about 
outcomes. Data collection sources were identified and included: encounter data; pharmacy 


data; and case management records. No sampling was conducted in this study. 


The data collection process for each measure was identified and well-defined. The follow-up 
information provided supported the assertion that accurate and valid data was collected. All 
data collection procedures were stated. The methodology was followed and included in the 
narrative. All data collection instruments are well-documented and the results included in the 
body of the study. A prospective data analysis plan for each measure was part of the study 


narrative. 


Interventions included mailings to the PCPs, Pulmonologists, and Allergists/Immunologists 
(pediatric and adult) treating the members identified. Mailings included a number of educational 
materials. Educational material was also mailed to members; this included a version of the 
Asthma Medication Guidelines. Analysis of all data collected was performed according to the 
documented plan. Data analysis material included baseline and updated information. All data 
was analyzed and described in detail including timeframes, type of analysis used, statistical 
significance testing, and results achieved. The overall results indicated positive results were 
achieved. The information provided indicated unexpected findings, which may create an 
additional study. The documentation provided also included new activities for the next project 
year. Emergency room utilization did decline for the members involved in the study; this was a 
result of the preventive medications administered and the availability of appropriate rescue 
medications. The narrative recognized variables that may have influenced the results, but 
asserted that the statistically significant changes that occurred were directly related to the 
impact of the study intervention. The results, including an analysis of significance testing, were 
documented and presented in the study narrative. The results presented also included the plan 


to follow the process for an additional year to obtain data to determine a sustained 
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improvement. The data, analysis, and information presented provide a high level of confidence 
that this project positively impacted the members included in the study. The MC+ MCO, in 
following this study should also develop strategies to incorporate these interventions into 


normal agency operations. 


The second PIP evaluated was “Improving Birthweight Outcomes.” The topic was supported by 
literature review and research information; however, the references for this information were 
not included. The MC+ MCO was operating the Guardian Angel Program for pregnant women 
who were MC+ Members. The MC+ MCO did not view this program as being as effective as 
necessary to impact high-risk pregnancies and premature deliveries. The topic of this study 
became “Reducing the number of high-risk pregnancies and premature deliveries will reduce the 
number of low birthweight infants born.” The goals of the study focused on identifying and 
correcting deficiencies in care in an effort to improve birth weight outcomes, thus improving the 
overall health of the infant population by reducing the probability of chronic lung disease, failure 
to thrive, potential developmental delays, and vision problems associated with low birth weight. 
This project focused on all pregnant MC+ Members. No specific portion of the population was 
excluded or omitted. The study question put forth was “Can prenatal case management, 
targeted at psychosocial factors, decrease the percentage of low birthweight infants?” The 
question appeared to be well-constructed and targeted. However the “psychosocial factors” or 


other issues leading to low birth weight were not specifically explored. 


Indicators were not well-defined. The only indicator listed was the percent of “‘Healthwave low 
birthweight infants.” Although the numerator and denominator specifications indicated “MC+ 
infants with birthweights < 2500 grams,” and “number of MC+ liveborn infants” respectively. 
Measured changes to be tracked were implied. The actual indicators that measure these 


outcome goals were not specifically delineated in the narrative provided. 


The study did include all pregnant MC+ members who were due to deliver after January I, 
2005. A variety of methods to correctly identify all pregnant members were included in the 
project description. It appeared from the description provided that the focus of the study was 
members in the Kansas City metropolitan area, and not necessarily those from the surrounding 
counties in this region. The MC+ MCO partnered with four in-home agencies to serve the 


pregnant population with the main intervention defined as a community-based approach to 
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providing services. Each of the agencies identified had different criteria for service eligibility. 
One only served Kansas. There was no clarification presented about how all enrollees needing 
these services might gain access to them. None of the contracted agencies provided services 


outside of Jackson County in Missouri. No sampling was conducted as part of this study. 


Data to be collected in the study was specified. The sources included information gathered 
from a focus group, OB registration forms, home visitor agency referral forms to be tracked in 
an Access database; CareEnhance clinical management software; and Amysis data. The narrative 
included information for baseline data collection, and a plan for future development. These 
sources and the data collection methodology were defined in the narrative. Most of the 
information to be gathered was to come from the MC+ MCO’s MMIS system. The 
methodology described appeared consistent with the data analysis plan. Tools to be used for 
tests of reliability and validity were discussed. The information provided did leave questions 
about timeframes and how home visitor data would be collected and tracked. The MC+ MCO 
did provide training about the Access database collection tool prior to study implementation. 
There was a prospective data analysis plan included that had sufficient detail to provide 


confidence in the methodology presented. 


The main intervention included in the project plan was referring all pregnant women, up to 20- 
30 weeks of pregnancy, to the home visitor programs for visits and face-to-face interactions. 
This intervention indicated a lot of promise for a positive impact on members. The substance 
of the intervention was not described. No follow-up or additional activities were described. 
Baseline data was established. Some preliminary data was also included in the narrative 
provided, but sufficient data for meaningful analysis was not yet available. This study concept has 
potential for a positive impact on members. The interventions could be more multi-faceted. 
Placing the success of the project on one intervention, in the hands a variety of agencies, all with 
their own eligibility criteria does not lead to confidence in production of measurable outcomes. 
lt appeared that the impact of all variables that could impact this approach was not considered. 
Some additional structuring and alternate interventions for member who are not eligible, or fall 
outside of the catchment area for the community-based agencies involved, would add credibility 


and access to members that may fall into the high-risk pregnancy category. 
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STRENGTHS 


|. The study topics selected were areas that have a strong potential to benefit and improve 
member services. 


2. The narrative and documentation included was improved. 


3. Data, updated narrative, and project analysis received at the site visit provided good data 
and measurement tools. 


AREAS FOR IMPROVEMENT 
|. The narrative should include enough information that a correlation can be made of the study 


question, interventions utilized, and outcomes. 


2. Ensure that interventions are comprehensive enough to produce real change and potential 
for improvement in services to members. 


RECOMMENDATIONS 
|. Utilize the Conducting Performance Improvement Projects protocol to ensure that all questions 


are answered and that the narrative is responsive to all areas that will be validated. 


2. Refine the format of the PIPs to ensure that enough detail is included to define the 
population studied and all details of the PIP process. 
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11.2 Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for FirstGuard Health Plan. FirstGuard Health Plan submitted the 
requested documents on December 9, 2005. The EQRO reviewed documentation between 
December 9, 2005 and February 27, 2006. On-site review time was used to conduct follow-up 
questions and provide feedback and recommendations regarding the performance measure rate 


calculation. 


Document Review 


The following are the documents reviewed by the EQRO: 


The Information Systems Capability Assessment (ISCA) submitted by FirstGuard Health Plan 
The Baseline Assessment Tool (BAT) submitted by FirstGuard Health Plan 

MetaStar’s NCQA HEDIS 2005 Compliance Audit Report 

Letters of communication between the EQRO and FirstGuard Health Plan 

FirstGuard Health Plan’s policies pertaining to HEDIS 2005 rate calculation and reporting 
FirstGuard Health Plan’s HEDIS Project Outline for 2005 

FirstGuard Health Plan HEDIS 2005 Medical Record Review Manual 

HEDIS 2005 Software Logs 

CRMS data warehouse tables - data field and definitions 

CRMS extract file build process documentation 

MC400 Data file extracts and print screens 

HEDIS analyzer error messages 

MOHSAIC data preparation process documentation 

HEDIS data file layout in the IDS SAS Warehouse 


The following are the data files submitted for review by the EQRO: 


FILE_|_ CHILD IMMUNIZATION. ENROLLMENT DATA txt 
FILE 2 CHILD IMMUNIZATION _ DENOMINATOR_AND_NUMERATOR txt 
FILE 3. CHILD IMMUNIZATION HYBRID.txt 

FILE_|_ DENTAL_ENROLLMENT_DATAI.txt 

FILE 2 DENTAL. DENOMINATOR_AND_NUMERATOR.txt 

FILE_|_ WELL_CHILD_ ENROLLMENT DATA.txt 

FILE 2 WELL CHILD. DENOMINATOR _AND. NUMERATOR.txt 

FILE 3 WELL CHILD HYBRID.txt 


Interviews 

The EQRO conducted on-site interviews with Susan Richart and Regina Webb, Eligibility 
Analyst, at FirstGuard Health Plan in Kansas City; and on the phone with Keith Hibbard, IT 
Director and Corey Waters, IT at Centene in St. Louis on Thursday, March 9, 2006. 


Subsequent interviews took place with Mr. Hibbard; Kelly Verhague, First Guard; Marsha 


Eversole, Stratum Med; and Carol Bostick, WebMD Business Services to complete First Guard’s 
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Information Systems Capabilities Assessment (ISCA) Report. They were in charge of explaining 
the process of calculating the HEDIS 2005 performance measures, as there had been a recent 
turnover in the HEDIS team. The objective of the visit was to verify the methods and processes 
behind the calculation of the three HEDIS performance measures. This included both manual 


and automatic processes of information collection, storing, analyzing and reporting. 


FINDINGS 

FirstGuard Health Plan used the Hybrid Method for the Childhood Immunization Status, 
Combination #2 and Well-Child Visits Measures. MCO to MCO comparisons of the rates of 
Adolescent Childhood Immunization Status, Combination #2, Well-Child Visits Measure, and 
Annual Dental Visit measures were conducted using two-tailed z-tests. For comparisons that 
were Statistically significant at the 95% confidence interval (Cl), the z-score (z), the upper and 


lower confidence intervals (Cl), and the significance levels (p > .05) were reported. 


The rate for the HEDIS 2005 Childhood Immunization Status, Combination #2 reported to the 
SMA and the State Public Health Agency (SPHA) by FirstGuard Health Plan was 55.72%. This 
was significantly higher than the statewide rate for all MC+ MCOs (28.17%; z = .60; 95% Cl: 
44.62%, 66.82%; n.s.). 


The rate for FirstGuard Health Plan for the HEDIS 2005 Well-Child Visits measure was 51.58%, 
significantly higher than the statewide rate for all MC+ MCOs (38.42%; z = .50, 95% Cl: 39.05%, 
64.11%; p > .95). The rate for FirstGuard Health Plan for the 2005 HEDIS Annual Dental Visit 
measure was 27.07%, comparable to the statewide rate for MC+ MCOs (29.76%, z = -.48; 95% 
Cl: 22.79%, 31.35%; n.s.). 


The following sections summarize the findings of the process for validating each of the 

performance measures in accordance with the Validating Performance Measures Protocol. The 
findings from all review activities are presented according to the EQRO validation activity, with 
the findings for each measure discussed within the activities as appropriate. Please refer to the 


tables in the main report for activities, ratings, and comments related to the Attachments. 
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Data Integration and Control 

For all three measures, FirstGuard Health Plan was found to meet all criteria for producing 
complete and accurate data (see Attachment V: Data Integration and Control Findings). There 
were no biases or errors found in the manner in which FirstGuard Health Plan transferred data 


into the repository used for calculating the HEDIS 2005 measures. 


Documentation of Data and Processes 

FirstGuard Health Plan Health Plan used NCQA-certified software from McKesson Inc, for the 
sampling and calculation of HEDIS measures. The EQRO was provided with a demonstration of 
the Health Plan Reporter (HPR), the application module for rate calculation, along with the 
CareEnhance Resource Management System (CRMS) data warehouse. Data and processes used 
for the calculation of measures were adequate (see Attachment VII: Data and Processes Used 
to Calculate and Report Performance Measures). FirstGuard Health Plan met all criteria that 


applied for all three measures. 


The information systems (IS) management policies and procedures for rate calculation were 
evaluated consistently with the Validating Performance Measures Protocol. FirstGuard Health 
Plan Health Plan was subject to a full Information Systems Capabilities Assessment (ISCA) this 
year due to a change in information systems after the Centene purchase. The EQRO found 
FirstGuard Health Plan to have most of the automated systems, management practices, data 
control procedures and rate calculation procedures required to assure that data is adequately 
captured, sorted, translated, analyzed and reported as mandated by the ISCA Review Protocol 
Appendix Z. The only area that the EQRO found to be lacking was FirstGuard Health Plan’s 
data quality monitoring; specifically FirstGuard Health Plan does not monitor quality beyond 
confirmation of value (i.e. record per batch). FirstGuard Health Plan should consider additional 


investigation to assure them of data quality that comes from vendors. 


Processes Used to Produce Denominators 
FirstGuard Health Plan met all criteria for the processes employed to produce the 
denominators of all three performance measures (see Attachment X: Denominator Validation 


Findings). This involves the selection of eligible members for the services being measured. 
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The HEDIS 2005 Childhood Immunization Status denominator file contained 41 | denominator 
cases. This is the minimum required sample size; there were no exclusions or replacements for 
the denominator of 41 |. Dates of birth and enrollment were within valid ranges, and there 


were no duplicate cases. 


The denominator file for the HEDIS 2005 Well-Child Visits measure contained a total of 41 | 
members, all members in the file were determined to be eligible for the measure. There were 


no duplicate entries and the dates of service were in the valid ranges. 


For the HEDIS 2005 Annual Dental Visit measure, there were a total of 18,573 eligible members 
(denominator) in the FILE_2 DENTAL_ DENOMINATOR_AND_NUMERATOR.-txt file. There 
were no duplicate members. The dates of birth and the dates of enrollment were both in the 


valid range. 


Processes Used to Produce Numerators 

All three measures included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-care visits, and asthma medications) as specified by the HEDIS 2005 criteria 
(see Attachment XIll:_ Numerator Validation Findings). Medical record reviews were conducted 


for the Childhood Immunization Status, Combination #2, and Well-Child Visits measures. 


For the HEDIS 2005 Childhood Immunization Status measure, FirstGuard Health Plan 
appropriately included administrative events from the State Public Health Immunization Registry 
(MOHSAIC). The dates of birth and dates of service were within valid ranges. Review of the 
administrative hits validated 233 of the |28 administrative hits reported in the files provided for 
review by the EQRO. This is an underestimate of 27.13%. Thirty (30) of 30 medical records 
requested for review were received; 6 records resulted in valid hybrid hits. Therefore, the 
medical record review validated 20 of the 101 hybrid hits reported. Based on the number of 
administrative and medical record review hits validated by the EQRO, the rate calculated was 
61.61%. The total estimated bias for the Childhood Immunization Status, Combination #2 
measure was a 5.89% underestimate of the rate. Although the rate of bias is not considerably 
significant, the EQRO found substantially more administrative hits than was reported on the 


DST. And although the hybrid hits reported by the MCO were not substantially supported by 
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the EQRO’s medical record review, the MCO had a small estimated bias due to the surplus of 


administrative hits found. 


For the HEDIS 2005 Well-Child Visits measure, the EQRO found 211 administrative hits, 
however only [52 hits were reported by FirstGuard Health Plan. All 30 medical record hits 
were requested and received by the EQRO for review. Twelve (12) of the records were 
validated for six or more well-child visits. Of the other 18 records, two (2) were found to have 
one visit, eight (8) were found to have four visits, and eight (8) were found to have five visits. A 
total of 235 hits were validated, although FirstGuard Health Plan only reported 212, for a rate of 


51.58%, a final underestimate of 5.60%. 


For the HEDIS 2005 Annual Dental Visit measure, a total of 18,573 unique members reported 
and validated. The dates of birth and service were within the valid ranges. The final rate was 


calculated to be 27.07% with no bias. 


Sampling Procedures for Hybrid Methods 

The Hybrid Method was used for the Childhood Immunization Status and Well-Child Visits 
measures. Attachment XIl; Impact of Medical Record Review Findings and Attachment XV: 
Sampling Validation Findings were completed for each of these measures. There were no 
exclusions calculated for the Childhood Immunization Status measure. None were allowable for 


the Well-Child measure. 


Submission of Measures to the State 

FirstGuard Health Plan submitted the DST for each of the three measures validated to the 
SPHA, (the Missouri Department of Health and Senior Services; DHSS) in accordance with the 
Code of State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) 


and the SMA Quality Improvement Strategy. 


Determination of Validation Findings and Calculation of Bias 
Table 85 summarizes the estimated bias in the rates reported by FirstGuard Health Plan, with 
the direction of bias. Although data processes and procedures for calculating both the 


Childhood Immunization Status, Combination #2 and Well-Child Visits measures were 
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adequate, the administrative rate calculated by the EQRO fell outside the 95% confidence 


interval reported by FirstGuard Health Plan. 


Table 85 - Estimate of Bias in Reporting of HEDIS 2005 Measures 
Measure Estimate of Bias Direction of Estimate 


Childhood Immunization Status, Combination #2 Underestimate 


Well-Child Visits in first 15 months of Life Underestimate 


Annual Dental Visit 





Final Audit Rating 
The Final Audit Rating for each of the performance measures was based on the findings from all 
data sources that were summarized in the Final Performance Measure Validation Worksheet for 


each measure. 


Table 86 - Final Audit Rating for HEDIS 2005 Performance Measures 
Measure Final Audit Rating 


Childhood Immunization Status, Combination #2 Substantially Compliant 

Well-Child Visits in first 15 months of Life Substantially Compliant 

Annual Dental Visit Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by the MCO. Not Valid = Measure deviated from State specifications such 
that the reported rate was significantly biased. This designation is also assigned to measures for which no rate was 
reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. FirstGuard Health Plan demonstrated significantly higher rates of Childhood Immunization 
Status and Well-Child Visits than the average for all MC+ MCOs. 


2. The rates for HEDIS 2005 Well-Child Visits was higher than the National Medicaid average 
for the measure. 


3. FirstGuard Health Plan used NCQA-certified software, CRMS/HPR of McKesson, for 
calculation of HEDIS measures. This software was reviewed and tested by NCQA auditors 
for source code verification. There were effective CRMS validation processes in place. 


4. There was effective use of the State Public Health Immunization Registry (MOHSAIC) data 
for calculation of the immunization measures. The data was directly loaded with mapping of 
data fields into the CRMS warehouse, which then processed it for reporting in the HPR 
module. 


5. FirstGuard Health Plan maintains close relationships with its vendors. These relationships 
are effectively supported by both FGHP policy and by appropriate staff allocation. 


6. FirstGuard Health Plan has a formal membership reconciliation process in place which 
increases accuracy and reliability of membership data and reporting drawn from that data. 
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7. 


FirstGuard Health Plan has a strong IS system in the use of Amisys and in the IS team at 
Centene. The manager of IS at Centene effectively demonstrated extensive knowledge of 
both the Amisys software and of the day- to- day operations at FirstGuard Health Plan. 


FirstGuard Health Plan, Centene, and the vendors, with whom the MCO has contracted for 
other data services, have effective industry best practice security measures in place. 


AREAS FOR IMPROVEMENT 


Data validation/integrity checks are lacking; no verification is done when data is transferred 
between systems. FirstGuard Health Plan assumes that a majority of verification is done by 
McKesson; this could not be verified. 


Data analysis should incorporate tests of statistical significance to assess whether the 
observed changes in rates are related to a specific intervention. 


FirstGuard Health Plan should also implement a plan though which its upper level managers 
can gain additional knowledge regarding the IS and its limitations. Such understanding, 
coupled with the demonstrated IS team understanding of FirstGuard Health Plan day-to-day 
operations would assist the MCO in reaching its quality control efforts. 


RECOMMENDATIONS 


Provide more comprehensive access to actual data used to calculate HEDIS rates. The on- 
site review was only of end-user level data, as the individuals with access to data in its 
original format are located at the Centene Headquarters office in St. Louis, MO. 


FirstGuard Health Plan should reduce its dependence on its contracted vendors as principals 
in data quality management. The MCO should look to random auditing, routine data 
sampling and other methods in addition to its use of log file and batch size to and from 
vendors to evaluate data quality. 
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11.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical fields? 


For the Medical claim type, there were 84,391 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 


The Outpatient Claim Type field was 00.00% complete, accurate and valid. 
3. The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 


4. The Outpatient First Date of Service field was 100.00% complete and accurate, and 99.99% 
valid. The 12 invalid dates of service ranged from |/120/2005 — 12/29/2005. 


5. The Outpatient Last Date of Service field was 100.00% complete and accurate, and 99.98% 
valid. The 19 invalid dates of service ranged from 4/01/2005 — 5/13/2005. 


The Outpatient Units of Service field was 100.00% complete, accurate and valid. 
The Outpatient Procedure Code field was 100.00% complete, accurate and valid. 


The Outpatient Place of Service field was 100.00% complete, accurate and valid. 


= 2 oS 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


10. Although the second through fifth Diagnosis Code fields are optional according to the 
Health Plan Record Layout Manual, they fell well below the 100% threshold established by 
the SMA for this validation. The second Diagnosis Code field was 34.67% complete, 
accurate and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). 
The third Diagnosis Code field was 0.55% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and 
invalid). The fourth Diagnosis Code field was 0.05% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code 
field was 0.00% complete, accurate and valid. 


For the Dental claim type, there were 9,923 encounter claims paid by the SMA for the period 


January |, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate 


and valid. 
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For the Home Health claim type, there were 5/77 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. 


eee YY 


The Outpatient Claim Type field was |100.00% complete, accurate and valid. 

The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 

The Outpatient First Date of Service field was 100.00% complete, accurate and valid. 
The Outpatient Last Date of Service field was 100.00% complete and accurate, and valid. 
The Outpatient Units of Service field was |00.00% complete accurate and valid. 


The Outpatient Procedure Code field was 74./0% complete and accurate, with 62.39% 
containing valid values. The following are the codes found in some of the 71 invalid fields. 





Frequency Code 
10 90378 

3 A4217 

24 A4221 

| A4222 

| Y9009 


The first and second Diagnosis Code fields were 100.00% complete, accurate and valid. 


Although the third through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA 
for this validation. The third Diagnosis Code field was 16.12% complete, accurate and valid 
(incomplete, inaccurate, and invalid). The remaining fields were blank (incomplete, 
inaccurate, and invalid). The fourth Diagnosis Code field was 7.45% complete, accurate and 
valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The fifth 
Diagnosis Code field was 6.07 % complete, accurate and valid. The remaining fields were 
blank (incomplete, inaccurate, and invalid). 


For the Inpatient claim type, there were 4,833 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 


= 2 





The Inpatient Claim Type field was |00.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 
The Admission Type field was 100.00% complete, accurate and valid. 


The Admission Date field was 100.00% complete and accurate, and 97.93% valid. There 


were 100 invalid dates ranging from | 2/02/2005 — |2/31/2005. 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 382 


Missouri MC+ Managed Care External Quality Review Section || 
Report of Findings — 2005 FirstGuard Health Plan 


2: 


|3. 


The Discharge Date field was 100.00% complete and accurate and 95.32% valid. There were 
226 invalid dates ranging from 04/01/2005 — 04/16/2005. 


The Bill Type field was 100.00% complete, accurate and valid. 


The Patient Status field was 100.00% complete and accurate and 99.81 %valid. There were 
nine invalid codes of “09”. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold 
for completeness, accuracy, and validity established by the SMA (99.63%, 77.90%, 59.78%, 
45.62%, respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


. The First Date of Service field was 100.00% complete and accurate, and 97.93% valid. There 


were 100 invalid dates of service ranging from | 2/02/2005 — 12/31/2005. 


. The Last Date of Service field was 100.00% complete and accurate, and 95.32% valid. There 


were 226 invalid dates of service ranging from 04/01/2005 — 04/16/2005. 


. The Revenue Code field was 99.94% complete, accurate, and valid. Three fields were blank 


(incomplete, inaccurate, invalid). 


The Units of Service field was 100.00% complete, accurate and valid. 


For the Outpatient Hospital claim type, there were 47,984 encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. 


oS Se 


The Outpatient Claim Type field was 100.00% complete, accurate and valid. 
The Recipient ID field was 100.00% complete, accurate and valid. 

The First Date of Service field was 100.00% complete and accurate, and valid. 
The Last Date of Service field was 100.00% complete and accurate, and valid. 
The Units of Service field was 100.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 100.00% complete, 99.99% accurate, and 99.91% 
valid. There were three entries that contained codes with fewer than the required 5-digit 
alphanumeric reference (one entry each of “981”, “9925”, and “9928”), resulting in the 
99.99% accuracy for this field. In addition, there were 3 entries of each invalid procedure 
code of “99601” and “99602” and 36 entries of “INVAL”. 


The Outpatient Revenue Code field was 100.00% complete, accurate and valid. 
The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the 
Health Plan Record Layout Manual, they fell well below the 100% threshold established by 
the SMA for this validation. The second Diagnosis Code field was 99.87% complete, 
accurate and valid. The remaining fields were blank (incomplete, inaccurate, and invalid). 
The third Diagnosis Code field was 24.01% complete, accurate and valid (incomplete, 
inaccurate, and invalid). The remaining fields were blank (incomplete, inaccurate, and 
invalid). The fourth Diagnosis Code field was 10.59% complete, accurate and valid. The 
remaining fields were blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code 
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field was 5.59% complete, accurate and valid. The remaining fields were blank (incomplete, 
inaccurate, and invalid). 


For the Pharmacy claim type, there were 70,413 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. All fields examined were |00.00% complete, accurate and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for FirstGuard Health Plan, an error analysis of the invalid entries 
was conducted for fields which were lower than the |00.00% threshold specified by the SMA. 
For the Medical claim type, the Outpatient First Date of Service, Outpatient Last Date of 
Service, and Outpatient Procedure Code fields contained invalid codes (see previous findings). 
For the Inpatient Claim type, there were invalid codes in the Admission Date, Discharge Date, 
and Patient Status fields, and three blank entries in the Revenue Code field. There were invalid 


codes in the Outpatient Procedure Code field for the Outpatient Hospital claim type. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, the Pharmacy claim 
type was consistent with the average for all MC+ MCOs, while Medical, Dental, Inpatient and 
Outpatient claim types were lower. The rate of Home Health claims was significantly higher for 
FirstGuard Health Plan than the average for all MC+ MCOs. These findings suggest a high level 


of completeness of data and at least moderate access to and utilization of services. 


What is the Fault/Match Rate Between State Encounter Claims and Medical 
Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 
types for the period of January |, 2005 through March 31, 2005 for medical record review. Of 
the 84,391 Medical encounter claim types in the SMA extract file for January |, 2005 through 
March 31, 2005, 100 encounters were randomly selected. Providers were requested to submit 
medical records for review. There were 76 medical records (76.0%) submitted for review. 


Encounters for which no documentation was submitted were unable to be validated. The match 
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rate for procedures was 51.0%, with a fault rate of 49.0%. The match rate for diagnoses was 


76.0%, with a fault rate of 24.0% 


What Types of Errors Were Noted? 

An error analysis of the errors found in the medical record review for procedure and diagnosis 
codes was conducted. For the diagnosis code in the medical record, the reasons for diagnosis 
codes not matching the SMA extract file were missing information (n = 22) or incorrect 
information (n = 2). The diagnosis code listed did not match the descriptive information in the 


record. 


For the procedure code in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = 12) and not enough information (n = 37). Examples of missing information 
included no code, codes listed that were not supported, or codes that did not match the 


procedure description. 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to 
the SMA encounter claim extract file due to the absence of an internal control number or 
crosswalk to compare the files. The plan was to compare MC+ MCO paid and unpaid claims to 
the SMA encounter claim extract file. MC+ MCOs were requested to submit data in a standard 
format (see Appendix 6) for the MC+ Managed Care Members represented in the encounter 


claim sample selected by the EQRO for validation. 


For purposes of the EQRO, FirstGuard Health Plan was able to submit files in the requested 
format, but the ICN codes submitted were not the same number of characters of the ICNs in 


the SMA extract files. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While the MC+ MCO did submit the data in the requested format, there are a number of ways 
to improve the data quality by improving the database system. One variable that is not 
currently represented is that of a unique line number. To match up specific lines of data (each 
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service provided within an encounter), this requires a unique number for each service provided 


for each member. 


STRENGTHS 


|. Encounter data was submitted to the EQRO in the requested format which allowed 
encounter validation for all claim types. 


2. The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


3. The critical fields evaluated for the Dental and Pharmacy claim types were |00.00% 
complete, accurate, and valid. 


AREAS FOR IMPROVEMENT 


|. For the Medical claim types, the Outpatient First Date of Service, Outpatient Last Date of 
Service, and Outpatient Procedure Code field contained invalid codes. 


2. Within the Home Health claim type, the Procedure Code field contained invalid entries. 


3. For the Inpatient claim type, the Admission Date, Discharge Date, Patient Status, and 
Revenue Code fields contained invalid data. 


4. For the Outpatient Hospital claim type, the Outpatient Procedure Code field contained 
invalid values. 


5. The match between the SMA encounter claims data and the medical records for diagnosis 
codes and procedure codes was not significantly lower than the average for all MC+ MCOs. 
The primary reasons for errors were missing and illegible information. 


RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the 
NSF/CMS 1500 file layout for the Outpatient Procedure Code and run validity checks after 
the programming of new edits. 


2. Ensure that Admission Date, Discharge Date, Revenue Code and Patient Status fields are 
complete and valid for the Inpatient (UB-92) claim types, and institute error checks to 
identify invalid data. 


3. Institute edits and error checks for the Outpatient Hospital claim type Procedure Code field 
(UB-92). 


4. For audit purposes, submit extract files for performance measures and encounter data in 
the requested file layouts with the requested documentation of files and include the SMA 
generated internal control numbers. 
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11.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract documents with the staff of the Division of Medical Services. On-site review 
time was used to conduct interviews with those who oversee the daily practices of the MCO. 
This ensures that documentation is developed and practices occur within the scope of the 


contract and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The following documents pertaining to FirstGuard Health Plan were reviewed prior to and at 
the on-site visit: 


The Division of Medical Services supplied: 
# State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

=» Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection of actions filed in the first quarter of 2005. 

# 2004 Annual Quality Improvement Program Evaluation 


Additional documentation made available by FirstGuard Health Plan included: 
# 2005 and 2006 Marketing Plan 
# FirstGuard Health Plan’s Organizational Chart 
# Missouri Medicaid Managed Care Program Primary Care Physician Attachment 
» Missouri Medicaid Managed Care Program Referral Physician Attachment 
» Subject Criteria Submission -MC+ Medicaid Managed Care Proposal 
» FirstGuard Health Plan MC+ Marketing Plan 
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*= Quality Project Summary — 2004 Documentation Form for Grievances and Appeals 
= Wooden Apple Award Criteria and Information 
= Member Grievance Category Trends 2004 
*" Quality Project Summary — ID Card Automation 
HIPAA Privacy Report automation 
State Payment Reconciliation 
CRMS Upgrade 
= ”Departments in the Spotlight” — Office Newsletters 


Interviews 
Interviews were conducted with the following group: 
Plan Administration 
Jean Rumbaugh, Plan President/COO 
William Pankey, MD, VP Medical Affairs 
Barbary Maloney, VP Development and Regulatory Affairs 
Celia Humphreys, Director, QI & Credentialing 
Sharon Taylor, Director, Customer Service 
Jackie Jones, Director, Utilization Management 


Susan Richart, Quality Improvement Project Manager 
Sandra Clausen, QI Program Manager 


Mental Health 
Claudia Sumrall, Clinical Manager 
Cindy Peterson, Service Center Director 


FINDINGS 

Enrollee Rights and Protections 

FirstGuard Health Plan continued to exhibit excellent practices in the area of Enrollee Rights 
and Protections. The MCO discussed their business transition to Centene Corporation during 
2005. FirstGuard Health Plan admitted facing a number of barriers to providing effective 
services, but were able to work through these in an effective manner. During the transitional 
period they received few member complaints. The MCO reported that the entire change 
remained seamless and transparent to members. Members, with very few exceptions, were 
unaware of the change. All telephone and contact numbers, interpreter services, and other 


methods of sharing information remained consistent for members. 


FirstGuard Health Plan‘s rating for compliance with Enrollee Rights and Protections regulations 


(100%) reflected their commitment to their members and to completing policy and procedure 
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as required. The MCO exhibited a firm commitment to ensure that members had access to 
healthcare services and obtained these services in an efficient and timely manner. FirstGuard 
Health Plan’s practice of treating members with respect and dignity during telephone and 

personal contacts remained unchanged. The MCO regularly provided training initiatives that 


focused on nondiscrimination, cultural competency, and minority inclusion. 


Table 87 - Subpart C: Enrollee Rights and Protections Yearly Comparison 
FirstGuard Health 


Federal Regulation EM 





438.100(a) Enrollee Rights: General Rule 
438.10(b) Enrollee Rights: Information Requirements 


438.10(c)(4,5) Language and Format: Interpreter Services 


438.10(d)(1)(i) Information Requirements: Format/Easily 
Understood 
438.10(d)(1)(ii)and (2) Information Requirements: Format Visually 


2 2 
2 2 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 
2 2 
2 2 


Impaired, and Limited Reading Proficiency ; Z 
438.10(f) Information for All Enrollees: Free Choice, etc. 2 2 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 5 5 
Plans 

438.10(i) Special Rules: Liability for Payment/Cost Sharing 2 2 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee Communications 2 2 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 2 2 
438.100(b)(3) Right to Services 2 2 
438.100(d) Compliance with Other Federal/State Laws 2 2 
Number Met 13 13 
Number Partially Met 0 0 
Number Not Met 0 0 
Rate Met 100.0% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Behavioral Health 

In October 2005, FirstGuard Health Plan changed subcontractors for behavioral healthcare 
services. They previously utilized Magellan Behavioral Health and now subcontract with 
Cenpatico, a subsidiary of Centene Corporation. The MCO provided assurances that all 
attempts were made to make this transition seamless and transparent to members as well. A 
sixty day transition period, requiring no pre-authorizations for services, was enacted. The MCO 
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ensured that the toll-free contact numbers for members remained constant. FirstGuard Health 
Plan facilitated meetings between Magellan and Cenpatico to discuss members who needed case 
management services, outreach services, and other specialized attention. Discussion between 
the behavioral health organizations continued throughout the transition phase of this 
subcontractor change. FirstGuard Health Plan and Cenpatico received no member grievances 
associated with the transition during this time. Cenpatico made the commitment to include all 
providers into their panel that were willing to make the change and the majority did contract 
with them. The MCO expressed the opinion that members were notified, but did not suffer 
therapeutically, as a result of required provider changes. FirstGuard Health Plan continued to 
hold oversight meetings with Cenpatico after completing the transition. The MCO continued to 
monitor Cenpatico’s performance and to meet monthly to ensure that member services were 


met. 


Cenpatico conducted case management services from a remote location. The MCO provided 
assurances that these case managers are familiar with and in close contact with local providers. 
FirstGuard Health Plan met two times a week to discuss specific cases and to identify new 
referrals to Cenpatico. The MCO’s stated goal was to formalize a collaborative process while 
working with the new subcontractor. FirstGuard Health Plan took monitoring activities 
seriously and watched cases closely to ensure that members received the attention and depth of 


services required. 


A number of FirstGuard Health Plan members were receiving in-home services, which were 
initiated since this subcontractor change occurred. Cenpatico staff explained that in-home 
services are consistent with their belief in the effectiveness of wrap around services. The BHO 
hoped that in appropriate member situations this type of service system could be built and 


delivered effectively. 


Quality Assessment and Performance Improvement 


Access Standards 

Both Centene Corporation and Cenpatico made a serious effort to contract with all providers 
previously within FirstGuard Health Plan’s network. The MCO’s efforts were successful in most 
cases and they considered these efforts a success. FirstGuard Health Plan acknowledged that 


they did experience payment issues during the transition, due to in health information system 
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deficiencies. The majority of the problems occurred with payment to non-participating providers 
as the new system would not allow payment. The MCO quickly resolved this problem by 
creating an internal process that by-passed these edits. This rectified most problems and 
allowed payment to occur. At one point, Centene Corporation allowed FirstGuard Health Plan 
to pay providers locally and later ensured that all invoices were properly completed, and that all 
required health information was included. At no point was there a denial of services to 
members. FirstGuard Health Plan historically had a strong relationship with providers and they 
maintained communication with affected providers during this period. Even when providers 
were unhappy with the MCO, they trusted that FirstGuard Health Plan would resolve these 
problems and correct payment errors. This gave providers the confidence needed to provide 


ongoing member services. 


FirstGuard Health Plan added staff in a number of areas, which positively affected their ability to 
respond to member and provider problems, complaints, and requests. The MCO related that 


this allowed them to recover quickly and effectively from the transitional issues. 


The MCO was using a new system called NurseWise for after-hours calls. The systems, which 
is physically located in El Paso, Texas, had bilingual staff available. The staff at NurseWise 
received training on the FirstGuard Health Plan system of care. With this system 24-hour nurse 
triage and nurse advice line was available. The MCO reported that this change provided several 
benefits to members. Members were assisted with information on providers, received help with 
scheduling if required, and were given information that allowed them to decide if the medical 
situation was urgent rather than emergent. Previously an administrator on call took after hours 


calls from members. 


FirstGuard Health Plan did implement a community advisory board during 2005 to assist the 
MCO in the assessment of member needs. The MCO also added a pharmacist to their staff and 
appointed this individual to a multidisciplinary team in an effort to add depth to the assessment 
process. This multidisciplinary team is utilizing improved data to complete trend analysis 


applicable to both members and providers. 


FirstGuard Health Plan continued to recruit new providers, but not as actively in previous years. 


They planned to renew recruitment efforts after the transition issues were completely resolved. 
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The rating for Access Standards (100%) reflects the effort made by FirstGuard Health Plan to 
have all policy and procedures completed and submitted to the SMA in a timely fashion. All 
information obtained and practice observed during the on-site review supported that FirstGuard 
Health Plan is in compliance with the MC+ Medicaid Managed Care contract requirements and 


the federal regulations. 


Table 88 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 










C—O FirstGuard Health 
Federal Regulation Plan 





438.206(b)(1)(i-v) Availability of Services: Provider Network 1 2 
438.206 (b) (2) Access to Well Woman Care: Direct Access 2 2 
438.206(b)(3) Second Opinions 2 2 
438.206(b)(4) Out of Network Services: Adequate and Timely 5 5 
Coverage 

438.206(b)(5) Out of Network Services: Cost Sharing 2 2 
438.206(c)(1)(i-vi) Timely Access 1 2 
438.206(c)(2) Provider Services: Cultural Competency 2 2 
438.208(b) Care Coordination: Primary Care 2 2 
438.208(c)(1) Care Coordination: !dentification 2 2 
438.208(c)(2) Care Coordination: Assessment 2 2 
438.208(c)(3) Care Coordination: Treatment Plans 2 2 
438.208(c)(4) Care Coordination: Direct Access to Specialists 2 2 
438.210(b) Authorization of Services 2 2 
438.210(c) Notice of Adverse Action 2 2 
438.210(d) Timeframes for Decisions, Expedited Authorizations 2 2 
438.210(e) Compensation of Utilization Management Activities 2 2 
438.114 Emergency and Post-Stabilization Services 2 2 
Number Met 15 17 
Number Partially Met 2 0 
Number Not Met 0 0 
Rate Met 88.2% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Structures and Operation Standards 
At the beginning of the transition to Centene systems, all data and providers were not properly 
transferred. Providers identified contracted procedures that were not in the system properly 
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and services were suddenly denied. All out-of-network providers and services were also 

denied. The MCO reacted to these problems proactively. Implementation staff and processes 
were put in place to assist in problem identification and resolution. Additional staff was added 
to enhance communication efforts with the provider community and to ensure that member 
services were not adversely affected. FirstGuard Health Plan administrators were very frank 
about the barriers and operational frustrations they encountered. They explained actions taken 
to resolve problems and how they worked through systematic and operational difficulties. The 
MCO made a concerted effort to regain the level of success they experienced in service delivery 
prior to the onset of the transition. This standard was restored in a matter of months, allowing 
FirstGuard Health Plan to refocus their efforts directly on provision of quality healthcare 


services for their members. 


FirstGuard Health Plan reported that they have experienced advantages of being associated with 
Centene Corporation. The MCO now has access to the Centene Fraud and Abuse system. 
Claim profiles were being reviewed and provider educational opportunities are identified. Some 
prescription abuse has been identified, most often related to pain management. FirstGuard 
Health Plan took action to assist members with appropriate services when this occurred. The 
MCO recognized opportunities within the Centene system, including a special investigations unit 
and additional software enhancements to capture and analyze data. FirstGuard Health Plan 


hoped to implement a number of enhancements within the next year. 


Ratings regarding compliance with the Structure and Operation Standards (100%) reflect 
FirstGuard Health Plan’s completion of all required policy and procedure. Although a number of 
operational issues were discussed during the on-site review, the MCO met these challenges 
proactively and resolved these issues in acceptable timeframes. It appears that FirstGuard 
Health Plan is fully compliant in meeting the requirements of the MC+ Medicaid Managed Care 


contract requirements and federal regulations. 
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Table 89 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


FirstGuard Health 


Federal Regulation Plan 


2 2 
72 2 
2 2 
ZZ 2 
2 2 
2 2. 
2 2 
72 2 
2 2 
1 2 
9 9 
1 1 
0 0 





438.214(a,b) Provider Selection: Credentialing/Recredentialing 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 
438.214(d) Provider Selection: Excluded Providers 


438.214(e) Provider Selection: State Requirements 


438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 
limitations 


438.56(c) Disenrollment Requested by the Enrollee 
438.56(d) Disenrollment: Procedures 

438.56(e) Disenrollment: Timeframes 

438.228 Grievance System 

438.230(a,b) Subcontractual Relationships and Delegation 


Number Met 
Number Partially Met 
Number Not Met 
Rate Met 90.0% 100.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Measurement and Improvement 

FirstGuard Health Plan was an active participant in the Kansas City Quality Improvement 
Consortium. This group developed locally accepted clinical guidelines for asthma and diabetes. 
The group was working on guidelines for treatment of obesity. The MCO disseminated and 
monitored application of these and other nationally accepted practice guidelines. The MCO 
continued one-on-one efforts with providers assisting in the acceptance of local and national 


guidelines. 


FirstGuard Health Plan continued operation of a Quality Assessment and Improvement 
program, which utilized a system of internal monitors, analysis, and evaluation. The focus was 
on improving the delivery of healthcare by MCO staff and providers. FirstGuard Health Plan’s 
staff, with experience in quality assessment, utilization management, and continuous quality 
improvement, performed the monitoring activities. Although much of 2005 was focused on 


transitional issues, they did not ignore Quality Assessment and Improvement activities. 
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Monitoring and analysis continued, incorporating changes that resulted from transition driven 


changes and requirements. 


FirstGuard Health Plan continued development and utilization of the Cactus software system for 
credentialing. Centene acknowledged the benefits of the Cactus system and was considering 
using it elsewhere. Not only does the system speed up the credentialing process, it produces a 
one page report for review of all information, rather than a set of laborious files. Organizational 
vendors and delegated entities, such as Children’s Mercy Hospital, are to be added to the 


database. 


FirstGuard Health Plan submitted two Performance Improvement Projects (PIPs) for validation. 
Specific details of these projects can be found in the appropriate section of this report. These 


PIPs were well-constructed and provided adequate information for validation. 


The MCO submitted all required information to complete the Validation of Performance 
Measures, as requested. FirstGuard Health Plan did begin using a new health information system 
during 2005, and they completed an Information System Capability Assessment (ISCA) in late 
2005, which is discussed in the appropriate section of this report. All encounter data requested 
was provided in the correct format. The details of each of these areas of validation can be 


reviewed within their specific sections. 


Ratings for the Measurement and Improvement section (100%) reflect that all required policy 
and procedure are in place. Observations and additional information obtained at the on-site 
review indicated that practice in this area is in compliance with the SMA contract requirements 


and the federal regulations. 
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Table 90 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison 


FirstGuard Health 


Federal Regulation Plan 





438.236(b)(1-4) Practice Guidelines: Adoption 2 2 
438.236(c) Practice Guidelines: Dissemination 2 2D 
438.236(d) Practice Guidelines: Application Z 2 
438.240(a)(1) QAPI: General Rules 2 2 
438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 1 5 
Improvement and PIPs 

438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 1 2 
438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 2 2 
438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare 5 5 
Needs 

438.240(e) QAPI: Program Review by State NA NA 
438.242(a) Health Information Systems 2 2 
438.242(b)(1,2) Health Information Systems: Basic Elements 1 2 
438.242(b)(3) Health Information Systems: Basic Elements 1 2 
Number Met 7 11 
Number Partially Met 4 0 
Number Not Met 0 0 


Rate Met 63.6% 100.0% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; I= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Grievance Systems 

Five member grievance files were reviewed for FirstGuard Health Plan. Six provider files were 
reviewed, all concerning complaints. There were some concerns about how these situations 
were handled, and the condition of the records. One member grievance indicated that the 
member attempted to make appointments, left a voice message, and no response was ever 
provided to the member. Corrective action occurred and an appointment was made. In 
another situation a member requested a new PCP because the current provider misplaced their 
file. The case manager involved did offer a new PCP, but education or other action with the 
provider or their staff was not documented. The third grievance involved a provider billing the 


member for a denied claim. It was learned that the denial was incorrect and the decision the 
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claim was paid. The correspondence to the member did not include language explaining that 
they were to be held harmless. The fourth grievance revealed that there had been a 
misunderstanding concerning physician office procedures. This was resolved, however, the 
letter to the member did not correspond to the situation surrounding the grievance. The letter 
referred to care received in an emergency room and stated that the situation would be 


investigated, but findings would be confidential. 


The content of files and the correspondence was inconsistent and often out of order. 
Terminology used was often confusing, including the use of terms such as complaint and appeal 


interchangeably. 


Six provider complaints were reviewed. Three of these complaints pertained to timely filing of 
claims. One was reviewed and upheld, while the other two were reviewed and overturned. 
The fourth complaint, which was upheld, involved a denied claim due to lack of prior 
authorization. The fifth complaint was an out-of-state provider who billed the out-of-state rate. 
The MCO paid this claim at the fee-for-service rate. The final complaint concerned denial of an 
inpatient admission, based on lack of medical necessity. The response letter was addressed to 
the member. The file contained no correspondence from the provider or member. 
Correspondence in these files was not always filed correctly, did not appear to contain all 


pertinent information, and was often confusing. 


Information about these records was shared with FirstGuard Health Plan at the on-site review. 
lt was recommended that they include some type of tracking form, particularly for 
correspondence received and sent, to clarify the chronological order of events, and to make the 


resolution easier to follow, for both internal and external reviews. 


FirstGuard Health Plan did complete all required written policy and procedures for Grievance 
Systems. Although the MCO had a Grievance System in place, their practice regarding 
maintenance of records and correctness of correspondence needed improvement. The rating 
for compliance with Grievance Systems (88.9%) reflects the practice witnessed at the time of 
the on-site review. The random selection of files that were reviewed came from the first 


quarter report submitted to the SMA. These files, which yielded both member and provider 
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records, created questions about the MCO’s monitoring of the grievance system and 


completion of correspondence regarding resolution for both members and providers. 


Table 91 - Subpart F: Grievance Systems Yearly Comparison 
FirstGuard Health 


Federal Regulation Plan 





438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2 
438.404(a) Grievance System: Notice of Action - Language and 5 5 
Format 
438.404(b) Notice of Action: Content 2 2 
438.404(c) Notice of Action: Timing 2 2 
438.406(a) Handling of Grievances and Appeals: General 5 5 
Requirements 
438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 
438.408(a) Resolution and Notification: Basic Rule 
438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 
438.408(d)(e) Resolution and Notification: Grievance and Appeals - 
2 2 

Format and Content of Notice 
438.408(f) Resolution and Notification: Grievances and Appeals - 

2 2 
Requirements for State Fair Hearings 
438.410 Expedited Resolution of Appeals 2 2 
438.414 Information about the Grievance System to Providers and 1 1 
Subcontractors 
438.416 Recordkeeping and Reporting Requirements 2 2 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 17 16 
Number Partially Met 1 2 
Number Not Met 0 0 
Rate Met 94.4% 88.9% 


Note: 0 = Not Met; I= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 
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Summary 

The 2005 healthcare service year presented FirstGuard Health Plan with a number of barriers to 
the provision of healthcare to their members. The MCO admitted that they experienced a 
number of challenges to maintaining the level of excellence that they expect their organization 
to produce. Struggles for administrative and front line staff were created with FirstGuard 
Health Plan’s transition to a new parent corporation, Centene, and the change in behavioral 
health organizations, from Magellan to Cenpatico. The organization admitted that the move to a 
new health information system was a massive effort, did not always occur as planned, and 
stressed their relationship with providers. Despite all the issues that occurred, the MCO 
maintained a strong focus on their members and meeting the members’ health care service 
needs. The transition itself and the problems encountered were transparent to members. 
FirstGuard Health Plan took a proactive approach to solving transition problems and built a 
stronger organization as the result of these efforts. FirstGuard Health Plan continued to accept 


recommendations for change throughout the process. 


The on-site review did identify some problems with maintenance of grievance records and 
correspondence. The MCO accepted recommendations made and was eager to rectify these 


issues. 


FirstGuard Health Plan completed required written policy and procedures to comply with the 
MC+ Medicaid Managed Care contract and the federal regulations. They continued to provide 
healthcare services that exceeded their strict contractual requirements in an effort to ensure 
that the children and families they serve have the best care available. FirstGuard Health Plan 
was able to maintain a positive relationship with providers during the transition period. The 
MCO implemented a community advisory committee to bring attention to unmet needs of both 
providers and members. They exhibited exceptional service in the areas of quality, access, and 


timeliness of services to members. 
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STRENGTHS 


2. 


The efforts made by FirstGuard Health Plan to ensure that the transition from Magellan 
Behavioral Health to Cenpatico Behavioral Health was transparent to members. 

The efforts made by FirstGuard Health Plan to maintain a positive relationship with their 
participating providers during the transition. The MCO discussed problems they 
experienced with billing and other changes. They were able to give examples of problem- 
solving techniques that allowed them to maintain or restore the relationship with their 
contracted providers. 

Improved internal processes, such as credentialing and fraud and abuse monitoring that are 
now available with the resources from Centene Corporation. 

Continued commitment of the FirstGuard Health Plan staff to produce and provide a quality 
product for their members and the community they serve. 

Completion of all required policy and procedures to be in compliance with the MC+ 
Medicaid Managed Care contract. 


AREAS FOR IMPROVEMENT 


The grievance and appeal files were difficult to evaluate and follow. The files did not contain 
all required correspondence, making it was difficult to ascertain if chronological 
requirements were met. 


2. Provide continued oversight to problem issues to ensure that local member needs and 
provider concerns are address in a timely and appropriate manner 

RECOMMENDATIONS 

|. Continue to development processes and services that are now available to FirstGuard 
Health Plan as part of Centene’s resource pool. 

2. Continue the attention the organization exhibits to identifying areas for quality improvement 
and responding to these issues quickly and efficiently. 

3. Include a tracking form in Complaint, Grievance, and Appeal files to ensure that all 
necessary information is included, and to highlight chronological decision-making process. 

4. Continue attention to the details required to maintain all policies and procedures in 


compliance with state and federal regulations. 
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12.0- blue ADVANTAGE’ 29 


Our most popular individual plan 


[2.1 Performance Improvement Projects 


The previous sections of the 2005 EQRO report present the purpose and objectives, technical 
methods, procedures for evaluation, and MCO to MCO comparisons of analyses, findings, and 
recommendations for each of the protocols based on the aggregate analysis. This section of the 
EQRO report summarizes MC+ MCO specific methods, procedures, findings, and 
recommendations for improving the quality, timeliness, and access to care for MC+ Managed 
Care Members. Please refer to the main report for detailed technical objectives, methods, and 


presentation of data that are referenced here for the MC+ MCO. 


METHODS 


Document Review 


Blue-Advantage Plus of Kansas City supplied the following documentation for review: 


= NCQA Quality Improvement Activity Form: Improving Care for Asthmatics 
= NCQA Quality Improvement Activity Form: Lead Testing Improvement Project 


Interviews 


Interviews were conducted with the project leaders for each Performance Improvement Project 
(PIP) by the EQRO team on March 9, 2006 during the on-site review, and included the 
following: 


Judy Brennan — Director State Programs BA+, Plan Administrator 
Janelle Martin — Quality Improvement Coordinator 

Dr. Bryan Sitzmann — Medical Director 

Wes Wadman — MHIP Coordinator 

Gwendolyn Nickles — Quality Improvement Project Director 
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Interviewees shared information on the validation methods, study design, and findings. Technical 
assistance regarding study design and presentation of findings was provided by the EQRO. The 
following questions were addressed: 


» What study questions were used? 

» What instruments were used for data collection? 

= How was the accuracy, consistency, and validity assured? 

» What interview instruments were used? 

=» Why were the projects valid for continuation and used as PIPs for this project year? 
» What findings were relevant to the MC+ population? 

= How was improvement analyzed? 

=» What are the conclusions about the effectiveness of the interventions analyzed? 


Several questions were presented during the on-site review and the MC+ MCO requested time 
to provide a response and additional information. Information was sent on April | 1, 2006. This 


information was considered in the final validation process. 


FINDINGS 

The first PIP evaluated was “Improving Care for Asthmatics.” This project was originated in 
1999. The additional information provided explained that the MC+ MCO chose to continue this 
project as a PIP because they were “‘still trying to address the high emergency room visitation 
rate” for members who are asthmatic and they had not reached the stated goals. Additionally, 
the MC+ MCO wanted to try and increase the participation of the membership into ’’the 
asthma program and link the number of members participating to improved functional status and 
symptom control.” The original information presented indicated that the MC+ MCO had 
experienced success with previous interventions. It appeared that this PIP was continued with 
no new activities. No information presented indicated any comparison of prevalence rates for 
people with asthma in the MC+ Member population to state or national statistics. No literature 
review or research was presented that would justify continuing these interventions, with no new 


activities or indicating that additional significant improvement was likely to occur. 
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The study presented clearly defined indicators: 

= Targeted goals for the number of inpatient admissions for the care of asthma; 

=» The number of emergency room visits for the care of asthma; 

= The percentage of MC+ MCO members identified with persistent asthma that were being 
prescribed medications acceptable as primary therapy for long term control of asthma. This 
indicator is based on HEDIS specifications. 

= Symptom frequency for nighttime awakenings in the program population. This measure is 
reported separately for adults (age 18 and over) and children. 

= The percentage of MC+ MCO members identified with asthma that filled prescriptions for 


greater than five canisters of a short-acting beta-agonist medication in a |2-month period. 


These indicators were well-defined and included methods of measurement. No detailed 
demographic characteristics were presented in the narrative. It could be inferred that no 
portion of the population was excluded from the study. The information presented made it 
difficult to distinguish between the commercial and MC+ population. A more detailed 
description of study participants would assist in specifying how the results related to the MC+ 


membership. 


There was no study question included in the original document presented. However, in the 
follow-up information provided the MC+ MCO provided the following: “Does engaging 
members in a disease state management program have an impact on decreasing emergency 
room utilization and/or symptom control as reported by the member?” Long term outcomes 
could be assumed, but the study narrative does not provide an explanation of how the 
interventions presented and the measures and outcomes are connected, or how any of these 


are related to the study question. 


The information provided creates some confusion about sampling. It does not appear that any 
actual sampling occurred. It appears that all eligible members “who could be reached,” were 
included in the personal interview, however, measure number C4 discusses a “convenience 
sample’ and “sampling size.” It describes the sample as “how many people we were able to 
contact during the quarter being measured.” The information does not explain what methods 
were used to contact members, the number of attempted contacts per member, or the number 


of contacts compared to the total members included in the measure. 
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The data sources described were specific. The additional information received explained that 
the methodology and interviewing results and the corresponding data collection methodology 
remained constant across all time periods studied. Information was provided by the contractor 
used to collect and compile the results of this study. The information presented outcomes from 
March 2000 through September 2005. It clarified that this information included all Blue Cross 
Blue Shield of Kansas City HMO and PPO members. It did include information specifically on 
the MC+ MCO members. The interview tool was not presented as it was considered 
confidential and proprietorial to AirLogix, Inc. The report that was provided did include the 
results of the questions asked. The subject of the questions could be inferred from the 


information provided, and the aggregate of the responses were presented. 


Barriers were identified throughout the original documentation, but they did not address how 
they led to any new strategies for improvement. The newest measures implemented were two 
years old. The referral source, information obtained through MC+ MCO claims data, and 
emergency room utilization reports were all updated. The contractor reinitiated the process of 
contacting members, providing an education plan, and working with physicians’ offices when 
deemed necessary. Large quantities of data from the sources identified were included, and 
analysis of the data was provided. The results were clarified with the report provided by 
AirLogix, Inc. The interpretation of the study findings indicated that the interventions were 
successful. It appears that the program allowed for measurement of sustained improvement as 
well as opportunity for additional improvement. The baseline methodology, which was 
modified, was clearly described and the implications for this change were understandable. The 
additional information provided, including the AirLogix, Inc. report, presented statistical 


significance testing and findings. 


This study, with the additional information, led to credible and valid findings. The study 
produced a high confidence for program improvement. The methodology should be 
implemented into regular practices of the MC+ MCO at this time and become part of their 


regular operations. 


The second PIP evaluated was “‘Lead Testing Improvement Project.” The topic was defined by 
the hypotheses presented, which was that “Lead testing rates will increase with education of 


providers and members about who should be tested and when the tests should be done”. The 
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need for improvement was explained in the narrative and was supported by review of the 
literature on this topic, as well as an examination of State and local issues. A broad range of 
enrollee issues were addressed by evaluating the effectiveness of provider and member 
education with the goal of increasing blood lead level testing rates. The narrative did not detail 
the participants included in this study, although the description of the measures consistently 
referenced “BA+" services and members. Eight indicators were presented with well-defined 
measures. The indicators defined benchmarks, sources, and baseline goals. The association 
between the indicators, measures, and outcomes was implied and apparent, but was not detailed 


in any narrative provided. 


No detailed description of the study population was included. The study focused on one county 
in the MC+ MCO service area, but this decision is not supported by the information or 
explanation included in the narrative provided. At the time of the on-site review MC+ MCO 
staff provided reasoning that Jackson County was identified because certain zip codes in the 
county had the highest percentage of individuals in high risk areas, and included a high number of 
disadvantaged families. The Department of Health and Human Services map concerning lead 
risk areas identified other counties in this MC+ Medicaid Managed Care region as high risk. The 


MC+ MCO did not respond to this observation. 


No study question was included in the original documentation submitted for this study. The 
MC+ MCO did provide additional information that included the following study question for this 
topic: “Will lead testing rates increase with education to providers and members about who 
should be tested and when the tests should occur?”” No sampling was conducted for this study. 
The sources of data were clearly identified and the study design included specific methods of 
collecting valid and reliable data over time. There was no prospective data analysis plan included 


in the narrative. 


Interventions included the educational material mailed to members, reminder letters for EPSDT 
testing to providers and members, supplies of filter paper to providers. This information was not 
well documented in the written information, but the details were provided at the time of the 


on-site review. The information provided a good description of analysis of barriers. 
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The findings for this study were not well described. Numerical and year-to-year comparisons 
were presented. Narrative documentation discussed some results and findings, but it was not 
clear how the accuracy of findings was documented. The analysis discussed initial and repeat 
measurements and identified that there were difficulties in tracking specific individuals because 
of the age factor and the changes that occur in the population that was studied. Figures 
presented accurately represented the number of lead tests conducted for each year presented. 
The narrative identified opportunities for improvement by restructuring the target groups and 


perfecting the methodology for measurement. 


The study had potential for producing credible findings. However, the information presented 
did not describe the effectiveness of the intervention with regard to the measurements 
completed. No impact on members was identified in the narrative. No significant improvements 
were identified in the findings presented. Any sustained improvement could not be determined. 
The format used to document the study findings made it difficult to validate the findings based 


on the requirements of the Validating Performance Improvement Protocol. 


STRENGTHS 


|. The MCO engaged in good topic selection in developing the Performance Improvement 
Projects. 


2. Each project had well-developed baseline data that assisted in the definition of success in 
defining the necessary interventions and in measuring PIP outcomes. 


3. The performance improvement projects included a focus on improving services to MC+ 
MCO members. 


4. The projects were based on sound research and literature studies although all information 
was not presented in a method that was easy to follow. 


5. Data collection was explained and understandable and in sufficient detail to complete the 
validation process, particularly with the additional information that was submitted after the 
on-site review. 
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AREAS OF IMPROVEMENT 


|. Ensure that the PIP narrative indicates the relationship between the study topic and 
question, interventions, the measures used, and expected outcomes. 


2. Include a clearly defined study question in the performance improvement project narrative 
that is related to the topic. 


3. Ensure that the population included in the study is well-defined and that this information is 
included in the narrative. 


RECOMMENDATIONS 


|. Utilize the performance improvement initiatives that are underway or proposed within BA+, 
such as the Complaint Process, as a non-clinical PIP for MC+. 


2. If utilizing on-going projects, ensure that a new activity is included and that the rationale for 
inclusion is provided. 


3. Utilize the Conducting Performance Improvement Project Protocol to assist in the process of 
project development and reporting. 


4. Include supporting narrative with submission of PIP, providing adequate explanation of the 
project, including graphs, surveys, or other instruments utilized to assist in the validation 
process. 
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12.2 Validation of Performance Measures 


METHODS 

This section describes the documents, data, and persons interviewed for the Validation of 
Performance Measures for Blue-Advantage Plus of Kansas City. Blue-Advantage Plus of Kansas 
City submitted the requested documents on December 9, 2005. The EQRO reviewed 
documentation between December 9, 2005 and February 27, 2006. On-site review time was 
used to conduct follow-up questions and provide feedback and recommendations regarding the 


performance measure rate calculation. 


Document Review 
The following are the documents reviewed by the EQRO: 


e The Baseline Assessment Tool (BAT) submitted by Blue-Advantage Plus of Kansas City 

e Ernst & Young’s NCQA HEDIS 2005 Compliance Audit Report 

e Letters of communication between the EQRO and Blue-Advantage Plus of Kansas City 

e Blue-Advantage Plus of Kansas City policies pertaining to HEDIS 2005 rate calculation and reporting 

e Blue-Advantage Plus of Kansas City Information Services (IS) policies on disaster recovery 

e Blue-Advantage Plus of Kansas City’s HEDIS implementation work plan and HEDIS committee agendas 
for 2004 

e Data warehouse validation procedures for the CRMS software 

e DB2 data warehouse models of the interim data warehouse 


The following are the data files submitted for review by the EQRO: 


e = childhood_immun_denominator.txt 
e = childhood_immun_numerator.txt 

e  dental_visit_denominator.txt 

e  dental_visit_numerator.txt 

e = Well_Child_denominator.txt 

e = Well_Child_numerator.txt 


Interviews 


The EQRO conducted on-site interviews with Phil Johnson, Director, Application Development; 
Judy Reynolds, Senior Application Developer; Barb Purdon, Project Lead; Darren Taylor, Vice 
President Enterprise Information and Access at Blue-Advantage Plus of Kansas City in Kansas 
City on Tuesday, March 8, 2006. This group was responsible for calculating the HEDIS 
performance measures. The objective of the visit was to verify the data, methods, and 
processes behind the calculation of the three HEDIS 2005 performance measures. This included 


both manual and automatic processes of information collection, storing, analyzing, and reporting. 
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FINDINGS 

Blue-Advantage Plus of Kansas City used the Administrative Method for calculation of the 
HEDIS 2005 Well-Child Visits in the First 15 Months of Life, Annual Dental Visit, and Childhood 
Immunization Status, Combination #2. MCO to MCO comparisons of the rates of Childhood 
Immunization Status Combination #2, Well-Child Visits, and Annual Dental Visit were 
conducted using two-tailed z-tests. For comparisons that were statistically significant at the 95% 
confidence interval (Cl), the z-score (z), the upper and lower confidence intervals (Cl), and the 


significance levels (p < .05) are reported. 


The rate for Blue-Advantage Plus of Kansas City for the 2005 HEDIS Childhood Immunization 
Status measure was 45.06%, comparable to the statewide rate for MC+ MCOs (28.17%; z = - 
V1, 95% Cl: 33.96%, 56.16%; n.s.). The rate for Blue-Advantage Plus of Kansas City for the 
HEDIS 2005 Well-Child Visits measure was 33.06%, comparable to the statewide rate for all 
MC+ MCOs (38.42; z = -.60, 95% Cl: 20.53%, 45.59%; n.s.). The rate for Blue-Advantage Plus of 
Kansas City for the 2005 HEDIS Annual Dental Visit was 33.80%, significantly higher than the 
statewide rate for MC+ MCOs (29.76%, z = .68; 95% Cl: 29.52%, 38.08%; p > .95). 


The following sections summarize the findings of the process for validating each of the 

performance measures in accordance with the Validating Performance Measures Protocol. The 
findings from all review activities are presented according to the EQRO validation activity, with 
the findings for each measure discussed within the activities as appropriate. Please refer to the 


tables in the main report for activities, ratings, and comments related to the Attachments. 


Data Integration and Control 

Blue-Advantage Plus of Kansas City used a NCQA-certified vendor application from McKesson, 
Inc. for calculation of rates for the HEDIS 2005 measures. The EQRO was provided with a 
process overview of the FACETS claims management system and a validation overview of the 
CareEnhance Resource Management System (CRMS) data warehouse. The EQRO was given a 
demonstration of the data flow and integration mechanisms for external databases for these 
measures, and provided with a layout of the data structure of the internally-developed data 
warehouse for storing interim data. For the three measures calculated, Blue-Advantage Plus of 


Kansas City was found to meet all criteria for producing complete and accurate data (see 
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Attachment V: Data Integration and Control Findings). There were no biases or errors found 
in the manner in which Blue-Advantage Plus of Kansas City transferred data into the repository 
used for calculating the HEDIS 2005 measures of Well-Child Visits in the First 15 Months of Life, 


Annual Dental Visit, and Childhood Immunization Status. 


Documentation of Data and Processes 

Data and processes used for the calculation of measures were adequate (see Attachment VII: 
Data and Processes Used to Calculate and Report Performance Measures). Blue-Advantage Plus 
of Kansas City met nearly all criteria that applied for the three measures validated. The two 
criteria that were partially met involved the use of statistical significance testing to document 
changes in performance over time and using this information to support claims of improvement 
or stability in performance over time. Blue-Advantage Plus of Kansas City has begun to utilize 
statistical testing, BA+ is partnering with Ernst & Young to best assess how to utilize the 


information that they obtain from the statistical analysis process. 


Processes Used to Produce Denominators 

Blue-Advantage Plus of Kansas City met all criteria for the processes employed to produce the 
denominators of the performance measures validated (see Attachment X: Denominator 
Validation Findings). This involves the selection of eligible members for the services being 
measured. Denominators in the final data files were consistent with those reported on the DST 
for the three measures validated. All members were unique and the date of birth ranges were 
valid. A total of 1,201 members eligible were reported and |,1 195 were validated for the Well- 
Child Visits measure. The 6 excluded from the measure by the EQRO were not enrolled in the 


MCO on their 15 month birthday, which is the anchor date for this measure. 


There were 15,634 eligible members reported for the denominator of the Annual Dental Visit 
measure, 15,446 were validated. The EQRO received files containing 16,268 records, 15,456 
were unique members, while 10 members were ineligible due to a gap in enrollment over 45 
days. 1,225 eligible members were reported for the Childhood Immunization Status measure, 
|,218 were validated. The seven (7) members excluded by the EQRO, were not enrolled in the 


MCO on their 2"4 birthday which is the anchor date for this measure. 
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Processes Used to Produce Numerators 
The measures validated included the appropriate data ranges for the qualifying events (e.g., 
immunizations, well-child visits, and dental visits) as specified by the HEDIS 2005 criteria (see 


Attachment XIll: Numerator Validation Findings). 


There were a total of 397 administrative hits reported and 361 validated for the HEDIS 2005 
Well-Child Visit measures. The dates of service and medical event codes (CPT and ICD-9 CM) 
were all within the valid ranges. The rate validated by the EQRO for Well-Child was 30.06%, 


with an observed bias of 3.00%. 


For the HEDIS 2005 Annual Dental Visit measure, a total of 5,285 administrative hits were 
reported and 5,282 validated by the EQRO. The final rate calculated by the EQRO was 33.79%, 


with no observed bias. 


There were a total of 552 administrative hits reported and | 15 validated for the HEDIS 2005 
Childhood Immunization Status measure. The dates of birth and dates of service were all within 
the valid range. The rate reported by the MCO was 45.06%, the rate calculated by the EQRO 
was 27.98% with bias of | 7.08%. 


Sampling Procedures for Hybrid Methods 
No sampling or medical record reviews were conducted or validated for the performance 
measures validated. Attachment XIl; Impact of Medical Record Review Findings and Attachment 


XV: Sampling Validation Findings do not apply to the Administrative Method. 


Submission of Measures to the State 

Blue-Advantage Plus of Kansas City submitted the DST for all three measures validated to the 
SPHA (the Missouri Department of Health and Senior Services: DHSS) in accordance with the 
Code of State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance Organizations) 


and the SMA Quality Improvement Strategy. 
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Determination of Validation Findings and Calculation of Bias 

As noted earlier, there was no bias found in the reporting of numerators, denominators, or 
rates of the Annual Dental Visit performance measure validated. The Well-Child Visits measure 
was overestimated, but was within the 95% confidence interval for the rates reported by the 
MCO. The Childhood Immunization Status measure was below the lower confidence level of 


the 95% confidence interval reported by the MCO. 


Table 92 - Estimate of Bias in Reporting of HEDIS 2005 Measures 


Measure ———C(W..__| Estimate of Bias _| Direction of Estimate 


Childhood Immunization Status, Combination #2 Overestimate 
Well-Child Visits in first 15 months of Life Overestimate 
Annual Dental Visit 





Final Audit Rating 
The Final Audit Rating for each of the performance measures was based on the findings from all 
data sources that were summarized in the Final Performance Measure Validation Worksheet for 


each measure. 


Table 93 - Final Audit Validation Rating for Performance Measures 

Measure Final Audit Ratince 

Childhood Immunization Status, Combination #2 Not Valid 

Well-Child Visits in first 15 months of Life Substantially Compliant 

Annual Dental Visit Fully Compliant 
Note: Fully Compliant = Measure was fully compliant with State specifications; Substantially Compliant = Measure 
was substantially compliant with State specifications and had only minor deviations that did not significantly bias the 
reported rate; A significant bias in the rate was defined as a number calculated by the EQRO that fell outside the 95% 
confidence interval of the rate reported by the MCO. Not Valid = Measure deviated from State specifications such 
that the reported rate was significantly biased. This designation is also assigned to measures for which no rate was 
reported; Not Applicable = No MC+ Members qualified for the measure. 





STRENGTHS 


|. Blue-Advantage Plus of Kansas City demonstrated significantly higher rates of Annual Dental 
Visits than the average for all MC+ MCOs. 


2. Calculation of the HEDIS 2005 Annual Dental Visit measure was fully compliant with 
specifications. 


3. Blue-Advantage Plus of Kansas City used NCQA-certified software (CRMS of McKesson) 
for the HEDIS rate calculations. This application was reviewed and tested by NCQA- 
certified auditors and has been found to accurately generate rates. There is an effective and 
documented CRMS validation process in place. The data for the calculation of HEDIS 
measures is loaded into the CRMS warehouse from the internally-developed central 
decision support (CDS) repository. 


ohare Performance Management Solutions Group 


ihens Group 


= senna m Bal A division of Behavioral Health Concepts, Inc. 415 


Missouri MC+ Managed Care External Quality Review Section |2 
Report of Findings — 2005 Blue-Advantage Plus of Kansas City 


4. Blue-Advantage Plus of Kansas City utilized many system checks to ensure data accuracy. 
Examples include production logs and control reports in the FACETS claims management 
system and the CDS repository. 


5. Blue-Advantage Plus of Kansas City had documented policies and procedures on calculation 
of HEDIS measures. There was an organization-wide, high-level policy for the calculation of 
HEDIS measures. There were associated project plans for calculation of HEDIS measures 
and effective disaster recovery policies in place, including live tests of disaster policies. 


6. There was an efficient data integration process in place. The pharmacy data was 
incorporated from Argus into the CDS repository, which also collects data from the 
FACETS claims management system. 


7. The information system had a unique business entity key for the calculation of the members’ 
continuous enrollment. This facilitates non-duplication of member data entry. 


8. Blue-Advantage Plus of Kansas City used a data warehouse system that incorporated an 
NCQA-certified vendor, MedMeasures, for HEDIS rate calculations. 


9. Blue-Advantage Plus of Kansas City had begun work with Ernst & Young to ensure the 
Statistical significance of HEDIS rates. 


AREAS FOR IMPROVEMENT 


|. Blue-Advantage Plus of Kansas City should utilize hybrid methods where HEDIS 
specifications recommend using the hybrid approach. 


2. Blue-Advantage Plus of Kansas City should continue its good efforts to full data systems 
integration. 


RECOMMENDATIONS 


|. Hybrid measures should be considered for all measures where the HEDIS specifications 
recommend a hybrid approach. Blue-Advantage Plus of Kansas City should compare the 
effects that the hybrid approach has on commercial rates to determine the possible increase 
in rates for MC+ HEDIS measures. 

2. Continue work with Ernst & Young to conduct and document statistical comparisons on 
rates from year to year. 


3. Continue the migration to a new data warehouse system with an integrated NCQA- 
certified vendor. This would provide better analytical processes for future performance. 
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12.3 Validation of Encounter Data 


FINDINGS 

The findings for the encounter data validation are organized according to the encounter data 
evaluation questions presented in the Technical Methods section for the encounter data 
validation in the aggregate report. Please refer to the main report for detailed objectives, 


technical methods and procedures for encounter data validation. 


What is the Baseline Level of Completeness, Accuracy, and Reasonableness 
of the Critical Fields? 


For the Medical claim type, there were 85,/07 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 

|. The Outpatient Claim Type field was 100.00% complete, accurate and valid. 

2. The Outpatient Recipient ID field was 100.00% complete, accurate and valid. 

3. The Outpatient First Date of Service field was |00.00% complete, accurate and valid. 


The Outpatient Last Date of Service field was 100.00% complete, accurate and 100.00% (with 
rounding) valid. Three invalid dates of service ranged from 4/20/2005--4/27/2005 


The Outpatient Units of Service field was |00.00% complete, accurate and valid. 


The Outpatient Procedure Code field was 100.00% complete, accurate and 99.96% valid. The 
31 invalid procedure codes consisted of 29 “B4035” and 2 “Y0043” entries. 


The Outpatient Place of Service field was 100.00% complete, accurate and valid. 
The first Diagnosis Code field was 100.00% complete, accurate and valid. 


Although the second through fifth Diagnosis Code fields are optional according to the Health 
Plan Record Layout Manual, they fell well below the 100% threshold established by the SMA 
for this validation. The second Diagnosis Code field was 48.76% complete, accurate and 
valid. The remaining fields were blank (incomplete, inaccurate, and invalid). The third 
Diagnosis Code field was 18.96% complete, accurate and valid (incomplete, inaccurate, and 
invalid). The remaining fields were blank (incomplete, inaccurate, and invalid). The fourth 
Diagnosis Code field was 8.69% complete, accurate and valid. The remaining fields were 
blank (incomplete, inaccurate, and invalid). The fifth Diagnosis Code field was 0.00% 
complete, accurate and valid. 


For the Dental claim type, there were 10,039 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. All of the fields examined were |00.00% complete, 


accurate and valid. 
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For the Home Health claim type, there were 485 encounter claims paid by the SMA for the 
period January |, 2005 through March 31, 2005. All of the fields examined were |00.00% 
complete, accurate and valid except the Procedure Code Field and the second through fifth 
Diagnosis Code fields. The Procedure Code field was 73.61% complete and accurate, and 
70.52% valid. There were |28 blank fields (incomplete, inaccurate, and invalid). There were |4 
invalid entries of “97001” and | invalid entry of “A4245”. The second, third, fourth, and fifth 
Diagnosis Code fields fell well below the 100% threshold for completeness, accuracy, and 
validity established by the SMA (72.65%, 51.55%, 24.33%, and 19.59%, respectively). The 


remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Inpatient claim type, there were |,250 encounter claims paid by the SMA for the period 
January |, 2005 through March 31, 2005. 


The Inpatient Claim Type field was 100.00% complete, accurate and valid. 
2. The Recipient ID field was 100.00% complete, accurate and valid. 
3. The Admission Type field was 100.00% complete, accurate and valid. 


4. The Admission Date field was 100.00% complete and accurate, and 95.52% valid. There 
were 56 invalid dates ranging from 12/01/2005 — 12/31/2005. 


5. The Discharge Date field was 100.00% complete and 99.76% accurate (with 3 entries of 
99999999"). Valid values were present 96.72% of the time. In addition to the 3 invalid 
“99999999” entries, 38 invalid dates ranged from 04/01/2005 — 04/15/2005. 


The Bill Type field was 100.00% complete, accurate and valid. 
The Patient Status field was 100.00% complete, accurate and valid. 


The first Diagnosis Code field was 100.00% complete, accurate and valid. 


— = at | 


The second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold 
for completeness, accuracy, and validity established by the SMA (83.76%, 62.48%, 46.64%, 
and 33.20%, respectively). The remaining fields were blank (incomplete, inaccurate, and 
invalid). 


10. The First Date of Service field was 100.00% complete and accurate, and 95.52% valid. There 
were 56 invalid dates of service ranging from | 2/01/2005 — 12/31/2005. 


ll. The Last Date of Service field was 100.00% complete and accurate, and 96.72% valid. There 
were 41 invalid dates of service ranging from 04/01/2005 — 05/18/2005. 


|2. The Revenue Code field was 99.92% complete, accurate and valid. There was one missing 
field. 


13. The Units of Service field was 100.00% complete, accurate and valid. 
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For the Outpatient Hospital claim type, there were 32,969 encounter claims paid by the SMA 
for the period January |, 2005 through March 31, 2005. All fields examined were 100.00% 
complete, accurate, and valid except for the second through fifth Diagnosis Code fields. The 
second, third, fourth, and fifth Diagnosis Code fields fell well below the 100% threshold for 
completeness, accuracy, and validity established by the SMA (54.45%, 25.76%, | 1.40%, and 


5.50%, respectively). The remaining fields were blank (incomplete, inaccurate, and invalid). 


For the Pharmacy claim type, there were 52,273 claims paid by the SMA for the period January 
|, 2005 through March 31, 2005. All fields examined were 100.00% complete, accurate and 


valid. 


What Types of Encounter Claim Data are Missing and Why? 

Based on the above analysis of the accuracy, completeness, and validity of the data in the SMA 
encounter claims extract file for Blue-Advantage Plus of Kansas City, an error analysis of the 
invalid entries was conducted for fields which were lower than the 100.00% threshold specified 
by the SMA. Dental, Hospital, and Pharmacy claim type critical fields examined were |00.00% 
complete, accurate, and valid. For Medical claims, Outpatient Last Date of Service and 
Procedure Code contain some invalid fields. For Home Health claims, the Procedure Code field 
contained some invalid data. The Admission Date and Discharge Date fields for the Inpatient 


claim type contained some invalid codes, and the Revenue Code field contained one blank entry. 


What is the Level of Volume and Consistency of Services? 

When comparing the rate of encounter claim types per 1,000 members, Blue-Advantage Plus of 
Kansas City demonstrated rates consistent with the average for all MC+ MCOs for the Medical 
and Dental claim types; and a significantly higher rate for Home Health encounter claims. There 
was a Significantly lower rate of Inpatient, Outpatient Hospital, and Pharmacy encounter claim 
types for Blue-Advantage Plus of Kansas City than the average for all MC+ MCOs. These 
findings suggest moderate to high access to care for Medical, Dental and Home Health Care 
services and lower access to Inpatient, Hospital, and Pharmacy services for Blue-Advantage Plus 
of Kansas City members. Another explanation might be that since preventative care appears to 


be readily available, BA+ members are healthier and do not require as many acute services. 
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To what Extent do the Claims in the State Encounter Claims Database 
Reflect the Information Documented in the Medical Record? What is the 
Fault/Match Rate between State Encounter Claims and Medical Records? 


To examine the degree of match between the SMA encounter claims database and the medical 
record, 100 encounters from each MC+ MCO were randomly selected from Medical claim 
types for the period of January |, 2005 through March 31, 2005 for medical record review. Of 
the 85,/07 Medical encounter claim types in the SMA extract file for January |, 2005 through 
March 31, 2005, 100 encounters were randomly selected. Providers were requested to submit 
medical records for review. There were 89 medical records (89.0%) submitted for review. 
Encounters for which no documentation was submitted were unable to be validated. The match 
rate for procedures was 65.0%, with a fault rate of 35.0%. The match rate for diagnoses was 


89.0%, with a fault rate of | 1.0%. 


What Types of Errors were Noted? 
An error analysis of the errors found in the medical record review for procedure and diagnosis 
codes was conducted. For the diagnosis code in the medical record, the reasons for diagnosis 


codes not matching the SMA extract file were missing information (n = I 1). 


For the procedure code in the medical record, the reasons for procedure codes in the SMA 
extract file not being supported by documentation in the medical record were missing 
information (n = 27) and not enough information to code (n = 8). Examples of missing 
information included no code, codes listed that were not supported, or codes that did not 


match the procedure description. 


What Problems are there with How Files are Compiled and Submitted by 
the MC+ MCO? 


lt was not possible to conduct the planned analysis of comparing MC+ MCO encounter data to 
the SMA encounter claim extract file because there was not a common key to match the files. 
The plan was to compare MC+ MCO paid and unpaid claims to the SMA encounter claim 
extract file. MC+ MCOs were requested to submit data in a format specified by EQRO (see 
Appendix 6) for the MC+ Managed Care Members represented in the encounter claim sample 


selected for validation. 
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For purposes of the EQRO, Blue-Advantage Plus of Kansas City was able to submit files in the 
requested format, but the ICN codes submitted were not the same number of characters of the 


ICNs in the SMA extract files. 


What are the Data Quality Issues Associated with the Processing of 
Encounter Data? 


While Blue-Advantage Plus of Kansas City did submit the data in the requested format, there 
are a number of ways to improve the data quality by improving the database system. One 
variable that is not currently represented is that of a unique line number. To match up specific 
lines of data (each service provided within an encounter), this requires a unique number for 


each service provided for each member. 


STRENGTHS 


|. Encounter data was submitted to the EQRO in the requested format which allowed 
encounter validation for all claim types. 


2. The critical field validation of all six claim types resulted in few fields under the SMA 
established threshold of 100.00% accuracy, completeness, and validity. 


3. The critical fields evaluated for the Dental, Outpatient Hospital and Pharmacy claim types 
were 100.00% complete, accurate, and valid. 


4. The match between medical records and the SMA encounter claims database for procedure 
codes was significantly higher than the average for all MC+ MCOs; and the match rate 
between diagnosis codes was consistent with the average for all MC+ MCOs. 


AREAS FOR IMPROVEMENT 


|. For the Medical claim type, Outpatient Last Date of Service and Procedure Code fields 
contained invalid entries. 


2. For the Inpatient claim type, there were invalid dates in the Admission Date and Discharge 
Date fields; also there was one blank Revenue Code. 


3. The Outpatient Procedure Code field in the Home Health claim type contained invalid fields. 


RECOMMENDATIONS 


|. Examine and revise as needed internal system edits for invalid procedure codes in the UB-92 
file layout for the Outpatient Procedure Code and Discharge Date fields, and run validity 
checks after the programming of new edits. 
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12.4 MCO Compliance with Managed Care Regulations 


METHODS 

Prior to the site visit, documentation was received and reviewed regarding the MCO’s 
compliance with the State contract. The External Quality Review Organization (EQRO) 
reviewed contract requirements with the staff of the Division of Medical Services. On-site 
review time was used to conduct interviews with those who oversee the daily practices of the 
MCO to ensure that documentation is developed and practices occur within the scope of the 


contract and in a manner that meets or exceeds federal regulations. 


A detailed protocol (BHC MCO Compliance Review Scoring Form) was utilized to ensure that 
all the elements of the federal regulations were addressed in the review process. Additionally, 
an interview tool was constructed to validate practices that occur at the MCO and to follow-up 
on questions raised from the document review and from the 2004 External Quality Review. 
Document reviews occurred on-site to validate that practices and procedures were in place to 


guide organizational performance. 


Document Review 


The Division of Medical Services supplied: 
» State of Missouri Contract Compliance Tool (including DMS responses and comments) 


The following documents were requested for on-site review: 

= Member Handbook 

= Provider Handbook 

# Provider Contracts 

= Grievance and Appeal Policies 

= Grievances and Appeals related to members were reviewed, as were Complaints, 
Grievances, and Appeal files for providers. These files were obtained from a random 
selection process of actions filed in the first quarter of 2005. 

= 2004 Annual Appraisal of the Quality Improvement Program 


Additional documentation made available by Blue-Advantage Plus of Kansas City included: 
#» 2005 and 2006 Marketing Plan and Educational Material Development Policy 
# Blue-Advantage Plus of Kansas City of Kansas City Organizational Chart 
= BA+ Brochures — English/Spanish versions 
= Complete Policy and Procedure Manuals 
# 2005 BA+ Member Survey Results 
= BA+ Oversight Committee Charter and Minutes 
= BA+ 2005 Report Card 
= BA+ Complaint Process “Race for Resolution” — Presentation and Materials 
# 2005 Well Aware Newsletters (Member) 
» 2005 Blue Speak Newsletters (Provider) 
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Interviews 
Interviews were conducted with the following group: 
Plan Administration 
Judy Brennan, Director, State Programs, Plan Administrator 
Dr. Loretta Britton, VP, Medical Director 
Sandy Wederquist, RN, Director, Medical Management 
Dr. Blake Williamson, VP. Senior Medical Director 
Shelly Bowen, AVP, Quality Management 


Tylisa Wyatt, Complaint Analyst 
Wes Wadman, MHIP Coordinator 


Mental Health 
Myron Unruh, AVP Clinical Operations 
Garth Smith, Director, Network Operations 
Lisa Woodring, Director, Prevention and Support Services 
Judy Brennan, Director, State Programs 


FINDINGS 

Enrollee Rights and Protections 

Blue-Advantage Plus of Kansas City continued to exhibit commitment and enthusiasm toward 
ensuring the member rights and protections are in place. Members were contacted quickly after 
the MCO learned of enrollment. A variety of continued contacts were made if initial attempts 
failed. Written information was provided in English or Spanish. If additional interpretive 


services were required, this was arranged for the member. 


Blue-Advantage Plus of Kansas City made changes in a number of processes to make service 
delivery easier for members. In January 2005 the MCO stopped requiring a primary care 
physician (PCP) referral for specialist care. The MCO made the policy and procedure review 
process less complex for ease in approval. Turnaround time on changes and requests for 


exceptions or review was occurring in a more timely manner. 


The MCO began use of a wellness van to participate in group events throughout the Kansas City 
area. The van made more than sixty appearances throughout 2005. Health information was 


provided and staff did screenings for lead poisoning, cholesterol, blood sugar, BMI and blood 


pressure. 
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A Radiology Advisory Committee was established with Doral Dental, a BA+ subcontractor for 
dental services, to evaluate high technology procedures offered in the communities surrounding 
Kansas City. Doral Dental released a series of privileging criteria to dental offices. There was 
an emphasis on advanced machinery in an effort to prevent excessive radiation exposure. New 
contracts were initiated utilizing the enhanced privileging criteria as the process was extended 
to dental practices where similar technology was available. Originally Doral Dental contracted 
with eighteen imaging centers. Three were unable to meet the new contract requirements. 
Both Doral Dental and Blue-Advantage Plus of Kansas City stressed that the new requirements 
are a benefit to members. Fewer contracted imaging centers have not limited member services 
as they provided a low volume of services to MC+ Medicaid Managed Care members. There 


was no negative impact on member access to services. 


Blue-Advantage Plus of Kansas City explained that they were transitioning from the traditional 
managed care medical management model to a care management model that provided more 
focused attention to members. The ultimate goal of the project is to empower members to be 
more responsible for improving their health status. The MCO began using a predictive modeling 
tool, Care Advance, to search through data and detect members who are at risk of needing care 
management services. Data used by the MCO included claims, pharmacy utilization, laboratory 
results, and self-reported information. When this process is fully operational follow-up contact 
will occur with all at-risk members detected. The members will receive prompts to make 
medical appointments, chronic disease treatment will be identified, and comparisons will be 
made to best practice guidelines. The MCO will produce assessments to submit to involved 
providers. Tutorials for chronic diseases, such as asthma and diabetes are available and 


providers will be able to use this information, as well as tracking patient information. 


During 2005, Blue-Advantage Plus of Kansas City had $700.000 in additional community service 
funding approved. Targeted initiatives included EPSDT, immunizations, and childhood issues. 
The MCO started an outcome based program called P.E. for Life, where they worked with 
schools to construct physical education areas. The MCO’s parent company, Blue Cross/Blue 
Shield of Kansas City donated funds to bring an “academy” type project to an urban school that 
did not have resources for this type of project. The MCO also assisted with the purchase of 
heart monitors to allow children to achieve improved health and fitness outcomes according to 


their own biometrics. Open houses were sponsored to promote this program. Before and 
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after-school programs were offered to assist with fitness objectives. Physical education grants 
continued to be available as start-up funding to allow additional districts to set up Physical 


Education for Life programs. 


The rating for Enrollee Rights and Protections (100.0%) reflects Blue Advantages Plus’s ability to 
have all policy and procedures submitted and approved by the SMA in a timely manner. The 
MCO also provided evidence of their practice throughout the on-site review process. It 
appears that the MCO is in compliance with all MC+ Medicaid Managed Care contract 


regulations and federal requirements. 


Table 94 - Subpart C: Enrollee Rights and Protections Yearly Comparison 


Blue Advantage + 


Federal Regulation 





438.100(a) Enrollee Rights: General Rule 2 2 
438.10(b) Enrollee Rights: Information Requirements 2 2 
438.10(c)(3) Alternative Language: Prevalent Language 2 2 
438.10(c)(4,5) Language and Format: Interpreter Services 2 2 
438.10(d)(1)(i) Information Requirements: Format/Easily 

Understood 7 2 
438.10(d)(1)(ii)and (2) Information Requirements: Format Visually 5 5 
Impaired, and Limited Reading Proficiency 

438.10(f) Information for All Enrollees: Free Choice, etc. 1 2 
438.10 (g) Information to Enrollees: Specifics/Physician Incentive 5 5 
Plans 

438.10(i) Special Rules: Liability for Payment/Cost Sharing 1 2 
438.100(b)(2) (iii) Enrollee Rights: Provider-Enrollee Communications 2 2 
438.100(b)(2)(iv,v) Rights to Refuse Services/Advance Directives 2 2 
438.100(b)(3) Right to Services 2 2 
438.100(d) Compliance with Other Federal/State Laws 2 2 
Number Met 11 13 
Number Partially Met 2 0 
Number Not Met 0 0 
Rate Met 84.6% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Behavioral Health 

Interviews occurred at the time of the on-site review with Blue-Advantage Plus of Kansas City 
and administrators from their Behavioral Health Organization (BHO), New Directions 
Behavioral Health. They reported on new programs that occurred during 2005, and responded 
to questions about follow-up to 2004 issues. The BHO reported on 2005 case management 
activities, such as work with all in-patient facilities to ensure that proper discharge planning 
occurred for members. This effort assisted in linking members with after-care services, such as 
those provided by area Community Mental Health Centers (CMHCs). The case management 
staff maintained a smaller, more manageable caseload that allowed for increased care 
coordination. New Directions did report that co-morbid cases increased during the past five 
years. The BHO reported that forty percent of the case management cases are Blue-Advantage 
Plus of Kansas City members, and that forty-one percent of these were identified by emergency 


room providers. 


As the result of efforts made by the SMA and the Missouri Department of Mental Health 
(DMH), a committee that reviews the use of psychotropic medications has been meeting for the 
past year. The BHO was an active member of this committee. Committee members included 
practicing psychiatrists and pharmacy representatives from the MCOs; the committee was 
chaired by Dr. Parks from the DMH. The committee reviewed pharmacy information and 
identified apparent misuses or duplication of pharmaceuticals. If abuse was detected through 
this process, the Blue-Advantage Plus of Kansas City member was contacted and prescriptions 
were monitored. All suspected pharmacy abuse was reported to the SMA on a quarterly Fraud 
and Abuse Report. The member was locked into a specific pharmacy for the period of one- 


year. 


New Directions Behavioral Health continued to jointly operate the PACT program with the 
Gillis Center. The PACT program has been in place for eight years. This program provided 
intensive interventions for members and their families, with follow-up services within the 
community. For example the program connected members and their family with their 
Community Mental Health Clinic (CMHC) for wrap around services or other beneficial 
interventions. These services, exceptional to the requirements of the MC+ Medicaid Managed 
Care contract, assisted members leaving in-patient care, and in some cases prevented in-patient 


care. Providing this type of support mechanism allowed the MCO to increase ambulatory 
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follow-up for members leaving in-patient services at the seven and thirty-day time frames. The 
follow-up numbers increased from 67% to 75.5% during the 2005 measurement period. New 
Directions met with providers and educated them about the importance of good discharge 
planning and about the availability of the PACT Program. The BHO reported that in 2003 their 
readmission rate was 10 percent. In 2004 and 2005 the readmission rate was reduced and 


sustained at 6-7 percent. 


New Directions reported that they established a practice of contacting all members presenting 
to emergency rooms three or more times in a quarter to ensure that any necessary follow-up 
services were available. Sharing information and reporting on outcomes was closely 


coordinated with Blue-Advantage Plus of Kansas City. 


The MCO worked with the BHO and sought the assistance of the Kansas City El Centra Group 
to inform both marketing and management staff about the large Latin population in the region. 
Spanish speaking consumers were brought into the staff training to inform staff of appropriate 
cultural considerations and barriers faced by this population in obtaining healthcare services. 
The group continued to work on written and verbal materials to ensure that all members have 


equal access to MCO basic information. 


Quality Assessment and Performance Improvement 


Access Standards 

Blue-Advantage Plus of Kansas City continued to have an extensive provider network available. 
The MCO reported that having regular access to orthopedic surgeons, neurologists and 
urologists was difficult. These specialists remain dissatisfied with the MC+ Medicaid Managed 
Care reimbursement rates. Blue-Advantage Plus of Kansas City did utilize specialists from their 
commercial network and reimbursed them at twenty percent over the Medicaid fee schedule. 


Customer Service staff continued active recruitment efforts for specialty medical providers. 


The MCO reported that their relationship with providers improved during 2005. Blue- 
Advantage Plus of Kansas City solicited participation from providers on an advisory committee 
that the MCO utilized for review of internal policies and activities. Physician complaints and 
member satisfaction surveys were used to trigger corrective actions and educational 


opportunities with providers. The MCO sent staff to provider offices to both monitor activities 
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with members and to assist Blue-Advantage Plus of Kansas City in defining problem areas 
reported. The MCO presented circumstances to the Advisory Committee to assist with 
problem solving. Blue-Advantage Plus of Kansas City placed forms and information on their web 


site for prior authorizations for provider convenience. 


Emphasis continued on active case management services within Blue-Advantage Plus of Kansas 
City, as additional attention was given to pregnant members. The MCO used a number of 
referral methods to identify pregnant members including the state health risk assessment, claims 
reports for first obstetrical visits, self-referrals, physician referrals, and information obtained at 
the Nurse Line. The MCO averaged 160 referrals each month, with eighty percent coming 
directly from providers. After a referral was received, the member was sent a packet of 
information, including health assessment forms (25% were returned). Any members identified as 
high-risk were sent to case management where contact was made and necessary follow-up 
services offered. The MCO collaborated with New Directions Behavioral Health for co-case 


management when this benefited members. 


Ratings regarding Access Standards regulations (100%) reflect that Blue-Advantage Plus of 
Kansas City submitted all required policy and procedures to the SMA for their approval. During 
the on-site review all practices observed indicated that the MCO made a concerted effort to 
ensure that they were compliant with the MC+ Medicaid Managed Care contract requirements 


and all federal regulations. 
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Table 95 - Subpart D: Quality Assessment and Performance Improvement: Access Standards Yearly 
Comparison 


Blue Advantage + 


Federal Regulation 





438.206(b)(1)(i-v) Availability of Services: Provider Network 
438.206 (b) (2) Access to Well Woman Care: Direct Access 


438.206(b)(3) Second Opinions 


438.206(b)(4) Out of Network Services: Adequate and Timely 
Coverage 


438.206(b)(5) Out of Network Services: Cost Sharing 
438.206(c)(1)(i-vi) Timely Access 

438.206(c)(2) Provider Services: Cultural Competency 
438.208(b) Care Coordination: Primary Care 

438.208(c)(1) Care Coordination: |Identification 
438.208(c)(2) Care Coordination: Assessment 

438.208(c)(3) Care Coordination: Treatment Plans 
438.208(c)(4) Care Coordination: Direct Access to Specialists 
438.210(b) Authorization of Services 

438.210(c) Notice of Adverse Action 


NO NON fF N 


438.210(d) Timeframes for Decisions, Expedited Authorizations 


438.210(e) Compensation of Utilization Management Activities 


NO NY NY NUN NN NY NY NY NY NY NY NY NY N N CN 


NO NY NN NY FN NY N NN N KN FEF 


438.114 Emergency and Post-Stabilization Services 
Number Met 14 
Number Partially Met 3 
Number Not Met 0 0 
Rate Met 82.4% 100.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 


determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


On 


Structures and Operation Standards 

Blue-Advantage Plus of Kansas City provided regular oversight to all subcontractors. The MCO 
met with New Directions Behavioral Health, their Behavioral Health Organization (BHO), at 
least monthly. Discussions with New Directions included access to services and completion of 
all studies to comply with URAC, NCQA, and MC+ Managed Medicaid regulations. 
Responsibility was delegated to New Directions for member grievance and appeal resolution 
and correspondence. This activity was monitored regularly by the MCO. Blue-Advantage Plus 
of Kansas City used Doral Dental, Inc. as their subcontractor for dental services. The MCO 
met with Doral Dental semiannually for oversight activities. The MCO used MTM as their 


transportation subcontractor. Although, Blue-Advantage Plus of Kansas City, like all of the 
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MCOs, identified problems with this subcontractor, they reported an effort by MTM to take 
complaints seriously and to utilize reliable service providers. The committee providing 

subcontractor oversight included a cross section of department heads who meet monthly to 
discuss pending issues and to create action plans for MCO staff or departments. In addition, 


Blue-Advantage Plus of Kansas City staff track this information on a daily basis. 


Blue-Advantage Plus of Kansas City implemented CareGuide QI software. This tool allowed for 
more efficient documentation of the Milliman Criteria and has allowed nursing staff to make 
more informed medical management decisions. Using this tool in collaboration with provider 
discussions allowed for the most appropriate authorization of inpatient services. The Milliman 
Criteria provided a guide for medical practice. However, the MCO also used specific practice 
guidelines from American College of Obstetricians and Gynecologists (ACOG) and the 
Academy of Pediatrics. All providers were encouraged to recognize best practices and follow 


nationally accepted guidelines. 


Ratings for compliance with Structure and Operation Standards regulations (100%) reflect that 
Blue-Advantage Plus of Kansas City has completed all policy and procedural requirements of the 
SMA. All practice observed during the on-site review supported that the MCO has made every 
effort to be compliant with both the MC+ Medicaid Managed Care contract requirements and 


federal regulations. 
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Table 96 - Subpart D: Quality Assessment and Performance Improvement: Structure and Operation 
Standards Yearly Comparison 


Blue Advantage + 


Federal Regulation 





438.214(a,b) Provider Selection: Credentialing/Recredentialing 2 2 
438.214(c) and 438.12 Provider Selection: Nondiscrimination 2 2 
438.214(d) Provider Selection: Excluded Providers 2 2 
438.214(e) Provider Selection: State Requirements 2 2 
438.226 and 438.56(b)(1-3) Disenrollment: Requirements and 5 5 
limitations 

438.56(c) Disenrollment Requested by the Enrollee 2 Je 
438.56(d) Disenrollment: Procedures 2 2 
438.56(e) Disenrollment: Timeframes 2 2 
438.228 Grievance System 2 2 
438.230(a,b) Subcontractual Relationships and Delegation 1 2 
Number Met 9 10 
Number Partially Met 1 0 
Number Not Met 0 0 


Rate Met 90.0% 100.0% 
Note: 0 = Not Met; 1= Partially Met ; 2 = Met 
Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 


Measurement and Improvement 

Emergency Room utilization was a topic of study during 2005. Blue-Advantage Plus of Kansas 
City was exploring this topic for a possible 2006 Performance Improvement Project (PIP). At 
the time of the on-site review some initial information was shared, but the MCO was in the 
process of completing a barrier analysis to make a final decision about implementing this topic as 
a PIP. Blue-Advantage Plus of Kansas City did submit two Performance Improvement Projects 
for validation that were conducted or completed during 2005. The detailed discussion regarding 


these PIPs can be found in the appropriate section of this report. 


Blue-Advantage Plus of Kansas City was involved in the community-based Kansas City Quality 
Improvement Consortium. The group developed clinical practice guidelines for diabetes and 
asthma. The group was continuing to work on obesity guidelines. The MCO continued to 
encourage all providers to use practice guidelines accepted by national organizations, as well as 
those based on local standards. The MCO used the Providers Office Guide and MCO 


newsletters to disseminate information about practice guidelines to the provider community. 
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Blue-Advantage Plus of Kansas City submitted all required information to complete the 
Validation of Performance Measures, as requested. The MCO continued to operate a health 
information system within the guidelines of that protocol. All encounter data requested was 
provided in the correct format. The details regarding these areas of validation can be reviewed 


within specific sections of this report. 


Ratings for the Measurement and Improvement sections were found to be (100%), which 
reflects that all required policy and practice meets the requirements of the MC+ Medicaid 


Managed Care contract and the federal regulations. 


Table 97 - Subpart D: Quality Assessment and Performance Improvement: Measurement and 
Improvement Yearly Comparison 






Blue Advantage + 


Federal Regulation 





438.236(b)(1-4) Practice Guidelines: Adoption 2 2 
438.236(c) Practice Guidelines: Dissemination 2 2 
438.236(d) Practice Guidelines: Application 2 2 
438.240(a)(1) QAPI: General Rules 2 2 
438.240(b)(1) and 438.240(d) QAPI: Basic Elements of MCO Quality 5 5 
Improvement and PIPs 

438.240(b)(2)(c) and 438.204(c) QAPI: Performance Measurement 1 2 
438.240(b)(3) QAPI: Basic Elements/Over and Under Utilization 2 2 
438.240(b)(4) QAPI: Basic Elements regarding Special Healthcare 5 5 
Needs 

438.240(e) QAPI: Program Review by State NA NA 
438.242(a) Health Information Systems 2 2 
438.242(b)(1,2) Health Information Systems: Basic Elements 1 2 
438.242(b)(3) Health Information Systems: Basic Elements 1 2 
Number Met 8 11 
Number Partially Met 3 0 
Number Not Met 0 0 


Rate Met 72.1% 100.0% 


Note: Regulation 438.240(e) refers to program review by the state. The regulation requires the state to review, at least 
annually, the impact and effectiveness of each MCO's quality assessment and performance improvement program. The 
regulation refers to the state QA & I program review process and is not applicable to External Quality Review of the 
MC+ Managed Care Program. This percent is calculated for the regulations that are applicable tot the MC+ Managed 
Care Program. 

0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols. 
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Grievance Systems 

Four member grievances and three member appeals were reviewed during the on-site review. 
Three of the member grievances concerned the transportation provider MTM. The issues 
concerned delays or failure to pick-up members. The drivers were counseled in all three cases. 
The fourth member grievance concerned a request by a member for a specific procedure, 
however, all appropriate paperwork was not submitted in a timely manner and the decision was 
upheld. Two of the member appeals were submitted by the provider on behalf of the members. 
One appeal requested payment for Synagis and the other requested payment for Enteral 
Feedings. Both denials were overturned. An appeal concerning a dental claim was originally 
upheld due to member ineligibility, but was later overturned upon a review of the member’s 


eligibility. 


Seven provider complaints were reviewed, three of which involved claims for inpatient hospital 
days. Upon review of medical necessity, all were approved. Two complaints involved payment 
for speech therapy; and both decisions were overturned with additional information. The final 
two complaints were reviewed and paid based on updated information and a correction of 


procedure codes. 


All files reviewed were conducted in a timely fashion and included enough information to 
understand the action taken and the basis of the decision made. All correspondence was sent in 
a timely manner. One letter concerning a decision in a member situation was worded 
incorrectly, which changed the meaning of the decision. This was shared with the MCO who 


agreed to send out a correct version immediately. 


Rating for compliance with Grievance System regulations (100%) remained complete as 
occurred in the 2004 review. The MCO takes pride in their Grievance and Appeal policy and 
procedures. All practice witnessed at the time of the on-site review, with the exception of one 
incorrectly worded letter, was also in compliance. The MCO agreed to correct the letter. 
They explained that when this type of error was identified, corrections were submitted to 


members and providers immediately. 
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Table 98 - Subpart F: Grievance Systems Yearly Comparison 


Blue Advantage + 


Federal Regulation 





438.402(a) Grievance and Appeals: General Requirements 2 2 
438.402(b)(1) Grievance System: Filing Requirements - Authority 2 2 
438.402(b)(2) Grievance System: Filing Requirements - Timing 2 2 
438.402(b)(3) Grievance System: Filing Requirements - Procedures 2 2 
438.404(a) Grievance System: Notice of Action - Language and 5 5 
Format 

438.404(b) Notice of Action: Content 2 2 
438.404(c) Notice of Action: Timing 2 2 
438.406(a) Handling of Grievances and Appeals: General 5 5 
Requirements 

438.406(b) Handling of Grievance and Appeals: Special 5 5 
Requirements for Appeals 

438.408(a) Resolution and Notification: Basic Rule 

438.408(b,c) Resolution and Notification: Grievances and Appeals - 5 5 
Timeframes and Extensions 

438.408(d)(e) Resolution and Notification: Grievance and Appeals - 5 5 
Format and Content of Notice 

438.408(f) Resolution and Notification: Grievances and Appeals - 5 5 
Requirements for State Fair Hearings 

438.410 Expedited Resolution of Appeals 2 2 
438.414 Information about the Grievance System to Providers and 5 5 
Subcontractors 

438.416 Recordkeeping and Reporting Requirements 2 2 
438.420 Continuation of Benefits while Appeal/Fair Hearing Pend 2 2 
438.424 Effectuation of Reversed Appeal Resolutions 2 2 
Number Met 18 18 
Number Partially Met 0 

Number Not Met 0 0 
Rate Met 100% 100.0% 


Note: 0 = Not Met; 1= Partially Met ; 2 = Met 

Sources: Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Monitoring 
Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A protocol for 
determining compliance with Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al. Final 
Protocol Version 1.0.; BHC, Inc., 2005 External Quality Review Monitoring MCOs Protocols 


Summary 

Blue-Advantage Plus of Kansas City has improved in meeting all policy, procedure, and practice 
areas of compliance with both the MC+ Medicaid Managed Care contract requirements, and the 
federal regulations. The MCO strengthened their programs, and engaged in a number of 
initiatives that served to improve the quality, access and timeliness of service to their members. 
Blue-Advantage Plus of Kansas City pointed to their member loyalty as proof of their focus on 
meeting member needs. The MCO continued to operate, expand, and create initiatives, several 
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in conjunction with the Behavioral Health Organization, that go beyond the strict requirements 
of their contract. These initiatives focus on prevention in an effort to avoid more intrusive 

treatment for members. Blue-Advantage Plus of Kansas City dedicated resources enabling staff 
to be responsive and supportive to members by ensuring that their healthcare needs were met 


in an effective and efficient manner. 


STRENGTHS 


|. Completion and approval of all required policy and procedures. 

2. Continuous development of new initiatives that enhance member services and utilize MCO 
resources, such as Care Advance, a project that uses MCO data to inform the MCO about 
member issues. 

3. The BA+ Complaint Process, “Race for Resolution,” a well constructed and important 
initiative that improved the MCO’s responsiveness and timelines to both member grievances 
and appeals, and provider complaints, grievances, and appeals. 

4. New investments in community initiatives such as the wellness van that participated in over 
sixty community events. 

5. Continued partnership with New Directions Behavioral Health and their exceptional 
initiatives, such as coordination of case management activities, the PACT, and PST programs. 


AREAS FOR IMPROVEMENT 


|. The parent company, BCBSKC, should assist the state programs section, BA+, in their focus 
on the MC+ program. In discussing many projects, BA+ often seems to get lost in overall 
activities. 


RECOMMENDATIONS 


|. Continue development of projects utilizing available resources and data to justify and assist 
in understanding member service needs. 

2. Continue development and use of products, such as CareAdvance, in predictive modeling 
and supporting empowerment of members to seek appropriate health interventions. 

3. Continue efforts to improve behavioral health services, such as monitoring inpatient 
facilities, completing proactive discharge planning, and aftercare services. 
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Introductions 
Orientation to Technical Methods and 
Objectives of Protocols 
Review of Information, Data Requests, and 
Timeframes 
Performance Measures 
Performance Improvement Projects 
Encounter Data Validation 
Compliance and Site Visits 
Closing Comments, Questions 
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Objectives and Technical Methods 
Validation of Performance Measures 
Validation of Encounter Data 
Validation of Performance Improvement Projects 
» MCO Compliance 
Requests for information and data 
List of BHC contacts for each protocol 


Presentation 
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Overview 





Protocol Activities 
Information and Data Requests 


Contact Persons 
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HEDIS 2005 Measure Validation for MC+ 
Well-Child Visits in the First 15 months 
of life 

‘Childhood Immunization Status 
Annual Dental Visit 
Administrative 
Hybrid method 


Review up to 30 medical records per 
measure sampled randomly 
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Performance Management 


Submission Requirements 
E aieart Main eceeen TE: / for PM Validation 


For each of the three measures: 
2005 HEDIS Audit Report 
Baseline Assessment Tool for HEDIS 2005 
BHO EGQRO Performance Measure Checklist (Method for Calculating HEDIS Measures; Table 1.xIs) 
List of cases for denominator with all HEDIS 2005 data elements specified in the measures 
se an appropriate delimiter fe.g., t for data that may contain commas or quotation marks). 
Data layout for the files will 06 provided in the data request, this data layout Must be Used to ensure Validity 
Listing of fields names and descriptions of fields (es. data dictionary) 
List of cases for numerators with all HEDIS 2005 data elements specified in the measures 
Use an appropriate delimiter fe.g., & for data that may contain commas or quotation marks). 
Data layout for the files will be provided in the data request, this data layout must be used to ensure validity 
Listing of fields names and descriptions of fields fi.e., data dictionary) 
List of cases for which medical records were reviewed, with all HEDIS 2005 data elements specified in 
the measures 
BHC will request MC Os to gather up to 30 records per measure, based on arandem sample, and MCO 
will Send copies 
Sample medical record tools used for hybrid methads for HEDIS 2005 measures and instructions. 


All worksheets, memos, minutes, dacumentation, policies and communications within the MCO and with 
HEDIS auditors regarding the calculation of the selected measures 


Policies, procedures, data and information used to produce numerators and denominators 
Policies, procedures, data used ta implement sampling 

Policies and procedures for mapping non-standard codes 

Others as needed 
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State encounter claim database 


Randomly selected encounters from medical 
claims, with service dates January 1, 2005 — 
March 31, 2005 


Review MCO supplied medical records for 
matching claims 

Match state and MCO claims databases for 
all encounters 
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Purpose and 
Objectives 


To assess the State encounter claim 
database quality (completeness, accuracy, 
and reasonableness). 

To validate the State encounter claims 
(paid) data against medical record 
documentation and obtain a fault rate. 


To examine the match between MCO claims 
(paid) and the State encounter claims 
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1. All State Encounter 







Claims, claims, 
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2005 
| (NM = 100 cases per MCO} 
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Critical fields will be examined for 
completeness (data in field), accuracy 
(correct type and length of data), and 
reasonableness (valid data for field) for each 
MCO. This will be conducted for all 
encounters in the specified time frame. 
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( Analyses: 2 


BHC will abstract the medical records and claims history/forms for 
each patient for the medical service provided during the entire time 
frame, enter Into a database, and determine the rate(s) of matches, 
omissions and commissions between the medical record and the 
State encounter claims for each MCO. Matches will be cases that 
are consistent on patient DCN, date of service, and diagnosis or 
procedure code. 

















State Medical 
Encounter Claims, 
January 1, 2005 - 
March 31, 2005 
(= 100 per MC) 













Medical Records 
(NM = 100 per MOCO} 






DCN-DOS-Code 
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Analyses: 3 





BHC will determine the rate(s) of matches, omissions 
and errors between the State encounter claims and 
MCO encounter claims for each MCO for the sample of 
selected cases. 











MC Encounter 
Claims (paid and 
Unpaid}, 
January 1, 2003 — 
March 31, 2005 
(NM = 100 per MCO} 









All State Encounter 

Claims, January 1, 

2005 — March 31, 
2U05 

(NM = 100 per MCO} 











DCN-DOS-Cade 
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Encounter Data 
/ Malidation Submission 


File 1: Provider mailing address and contact information for sampled 
claims (service dates January 1, 2005 to March 31, 2005). This will be 
used for validation of the State medical encounter claims database 
against the medical record. 


File 2: All inpatient encounters from January 1, 2005 to March 31, 2005 
for selected MC+ members, with detailed provider information. This 
should be in the layout specified by BHC in the Encounter Data 
Submission Instructions. 


File 3: All outpatient encounters (Outpatient, Medical, Dental, and 
Home Health) from January 1, 2005 to March 31, 2005 for selected 
MC+ members, with detailed provider information. This should be in the 
layout specified by BHC in the Encounter Data Submission Instructions. 


File 4: All pharmacy encounters from January 1,2005 to March 31, 
2005 for selected MC+ members, with detailed provider information. 
This should be tn the layout specified by BHC in the Encounter Data 
Submission Instructions. 









| Performance Management 
Solutions Group 
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NOTE: “unpaid claims” are those claims that the MCO denied for payment, unpaid claims do 
not include claims paid via a capitation plan. 
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Encounter 
Encounter sample provided to MCO 
MCO to develop Files 1 (2 weeks from receipt of sample) 
MCO to develop Files 2, 3, 4 (6 weeks from receipt of 
sample) 
MCO to submit medical record request to providers (1 week 
from development of File 1) 
MCOs to ensure providers supply medical records to BHC (4 
weeks from submission of request to providers) 

| HEDIS 

Medical record samples requested from MCOs 
for 2 possible hybrid measures (N < 30 per 
measure; 4 weeks) 
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Medical Record 


7 Perform ance Management 






x _ mewn _/ Reviews (Cont'd 
MCO will request and obtain Medical Records 
from providers 
Letter from Sandra Levels 
Instructions for submitting records 
Encounter claim supporting information, dates, 
notes, claims information 
Explanation of Confidentiality, storage of files 
Explanation of HIPAA, Business Associate 
wee greement, Health Oversight Authority 
ee 
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Medical Record 












ent Reviews (Cont'd 
Reviewed and abstracted by experienced and 
certified medical coders 
Standard abstraction tools 
Matching DCN, Date of Service, Diagnosis 
Code, Procedure Code 

| 
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Validation of Performance 






ng Certermanon Me a Sa ea : : : 
| “Mostascnitiomcmmestt. / [improvement Projects 


Two Performance Improvement 
Projects underway In 2005 
“One clinical 


One non-clinical 
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Validation of Performance 
_Improvement Projects and 
Submission Requiremen 






PIP Checklist Elements 


oN 





Project narratives, baseline measures, methods, interventions, and planned analyses. 
Examples of information are contained in the CMS protocol, Validation of Performance 
Measures[1] 

Phase-in/timeframe for each phase of each PIP[1] 

Problem identification 

Hypotheses 

Evaluation Questions 

Description of intervention(s) 

Methods of sampling, measurement 

Planned analyses 

Sample tools, measures, surveys, etc. 

Baseline data source and cata 

Cover letter with clarifying information 

Raw data files (if applicable, on-site) 

Medical records or other original data sources (if applicable, on-site) 

Additional data as needed 


ta U.S. Department of Health and Human Services, Centers for Medicare & Medicaid 
services (2002) VALIDATING PERFORMANCE IMPROVEMENT PROJECTS A protocol 


for use in Conducting Medicaid External Quality Review Activities: Final Protocol Version 
1.0 May 1, 2002 
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MCO Compliance 
-Follow-Ur 


Enrollee Rights 





nce Managem ‘ 
ons Group 
Division of Behavioral Health Concepts, inc. SH C., 








Grievances and Appeals 
Quality Improvement 


Submission Requirements TBD 
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a Division of Behavioral Heatth Concepts, inc. BHC, 


‘texeenarceucrmernes ~~ Site Visits 


Target for February, 2006 
MCO Compliance Reviews follow-up 
On-site activities 
Information Systems Capability Assessments 
_ Performance Measure Validation 


Performance Improvement Project Validation 
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([samreacere ~ ) Final Report 


| Se aces 
_ ——— _ 
MCO to MCO Comparisons: 
Encounter data match/fault rates for 
diagnoses and procedures 
Performance Measure audit findings 
and rates 
Performance Improvement Project 
element compliance 
MCO Compliance follow-up 
eee vision of Bonavioal Health Concept, nc 
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Performance Management 


BHC Team and Coordination 








( { | Solutions Group 
4) a Division of Behavioral Health Concepts, inc. BHC, 
BHC Contact 
Behavioral Health Concepts, Inc. 
2? 16 Forum Glyd., Suite 4a 
oe Columbia, MO 65203 
Protocol/ Activity Tel. 573-446-0405 Fax 573-446-1816 MCO Contact 
se | : Ms. Amy McCurry 
Performance Measures Priiett Ditentot 
(HEDIS 2005) aMccurnd@pmsdinfo.com 
Ms. Mona Prater 
Assistant, Project Director 
Mpraten@pmsginto. com 
| | Ms. Marva Chumak 
Performance : Research Associate 
Improvement Projects mchumak@bhcinfo.com 
Mariya Chumak 
Encounter Data mchurmak@ebhcinfo com 
i Mona Prater 
MCO Compliance mprater@omsginfo.com 
Amy WeCurry 
amecur msqinto com 
2 ., Mona Prater 
Site Visits mprater@omsginfo com 
f Mariya Chumak 
Medical Records mchumakiabhcinfo corm 
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Appendix 2 — PIPs Checklist/Worksheets 


Performance Improvement Project Validation Worksheet 


| 
\Performance Improvement Projects. Answer all questions for each | 
lactivity. Refer to protocol for detailed information on each area. | 


ID of evaluator Mona Prater Date of evaluation 


or-Turtelele-lelullom Grice) gurl-ialela 


MCO/PIHP Name or ID Project Leader Name Telephone Number 


Name of the Performance Improvement Project 





Dates of Study Date Study Initiated 


Type of Delivery System (check all that apply) 








l@| Staff Model ‘= Network ‘=| Director IPA 

i=) IPA Organization ‘= MCO ‘=| PIHP 
Number of Medicaid Enrollees Number Medicare Enrollees in 
in MCO or PIHP* MCO or PIHP 
Number of Medicaid Enrollees Total Number of MCO or PIHP 
in the Study —_—__— Enrollees in Study 
Number of Members in Study Population of Members in 


Sample Frame 





Number of MCO/PIHP Number of MCO/PIHP 

primary care physicians specialty physicians 
Population of physicians in Number of physicians in study 
sample frame 


Note: DK = Don't Know; NA = Not Applicable 


* Source: Missouri Medicaid Management Information System COLD Reports, State Session MPRI 
Screen, Revised June 25, 2004. Enrollment totals include enrollees with a future start date; 1115, 1915b, 
and Title XXI enrollees as of June 25, 2004. 
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Activity 1: ASSESS THE STUDY METHODOLOGY 
Step 1. Review the selected study topics(s) 









1.1 The topic was selected through data collection je) Met ‘=| Partially met lm! Not met 
and analysis of comprehensive aspects of enrollee : 
needs, care and services. ‘®@| Not applicable \@ Unable to determine 
Topic or problem 
statement: 
Clinical 

|_| Prevention of an acute or chronic condition __| High volume services 

|_| Care for an acute or chronic condition __| High risk coniditions 
Nonclinical 

| | Process of accessing or delivering care 
Comments 
1.2 MCO's/PIHP's PIPs, over time, addressed abroad |» Met ‘| Partially met [im Not met 
spectrum of key aspects of enrollee care and services. : 5 ; 

\@ Not applicable ‘| Unable to determine 


Project must be clearly focused on identifying and correcting deficiences in care or 
services rather than on utilization or cost alone. 


Comments 
1.3 MCO's/PIHP's PIPs over time, included all (ml Met (®) Partially met (i=) Not met 
enrolled populations: i.e., did not exclude certain 
enrollees such as those with special health care © Notapplicable ‘© Unable to determine 
needs. 

, vss Payor 
Demographic description Age i. y 

: — ce 

of MC+ population Paid MC+ 
Comments Commercial 
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Step 2: Review the study question(s) 


2.1 Study question(s) stated clearly in writing. ‘wl Met (i) Partially met \@| Not met 


'@ Not applicable 'S§\ Unable to determine 
Study question(s) as 
stated in narrative: 





Comments 


Step 3. Review selected study indicators(s) 











3.1 The study used objective, clearly defined, = Met (im) Partially met (wl Not met 
measurable indicators. ; , 
‘© Not applicable \™@| Unable to determine 
Indicators (list): 
Comments 
3.2 The indicators measured changes in health status, {§| Met _{ii Partially met im) Not met 
functional status or enrollee satisfaction; or process of : 
care with strong association with improved outcomes. © Not applicable \@ Unable to determine 
Long term outcomes implied or stated: ®) Yes (=| No 
Health status: Satisfaction (members): 
Functional status: Satisfaction (providers): 
Comments 
© 2004 Behavioral Health Concepts, Inc. Page 3 of 11 
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Step 4: Review the identified study population 

















4.1 MCO/PIHP clearly defined all Medicaid enrollees e| Met (| Partially met ‘m| Not met 
to whom the study questions and indicators are . 
‘elevaint. | Not applicable im) Unable to determine 
Demographic description of Age Race — + it 
MC+ population sampled SSS iain Heccnaneinl 
Did it include: 
1115 lm Yes \™| No ®| Unable to determine =) NA 
1915b lm! Yes ‘=| No ©) Unable to determine m| NA 
Children in state custody ‘ml Yes ‘@| No =| Unable to determine m| NA 
Consent Decree (Western) \@| Yes ‘=| No =| Unable to determine im) NA 
Comments 
4.2 If the MCO/PIHP studied the entire population, did “| Met (| Partially met ‘| Not met 
its data collection approach capture all enrollees to 
whom the study question applied? "| Not applicable ‘= Unable to determine 
Methods of identifying —_{ | utilization data _| referral 
participants: _- ; ‘ ‘ 
|_| self-identification _| other 
Comments 


Step 5: Review sampling methods 


5.1 Sampling technique considered and specified the ‘= Met ‘=| Partially met ‘=| Not met 
true (or estimated) frequency of occurrence of the | 
event, the confidence interval to be used, and the ‘©, Not applicable ‘= Unable to determine 


margin of the error that will be acceptable. 


Previous findings from: 
| literature review | | baseline assessment of indices — Other 


Comments 
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5.2 The MCO/PIHP employed valid sampling ‘© Met (m) Partially met (m| Not met 
techniques that protected against bias. ‘ 
© Not applicable \@ Unable to determine 

The type of sampling used: 

|_| Probability | Nonprobability |_| Random __ Simple | Stratified 

| | Convenience | Judgment |_| Quota " Cluster 

Comments 

5.3 Sample contained sufficient number of enrollees. «) Met ‘@ Partially met ‘m] Not met 

®) Not applicable \@ Unable to determine 
N of enrollees in sampling frame N of sample 


N of participants (i-e., return rate) 


Comments 


Step 6: Review data collection procedures 


6.1 Study design clearly specified the data to be ‘@ Met \m| Partially met = Not met 
collected. ; x ¥; 

\@ Not applicable = Unable to determine 
Comments 
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6.2 The study design clearly specified the sources im, Met \m| Partially met \m| Not met 
of data. 
\™ Not applicable ‘=| Unable to determine 
Source of data: 
|_| Member Claims |_| Provider | Other 
Comments 
6.3 The study design specified a systematic method =) Met \@| Partially met 'm| Not met 
of collecting valid and reliable data that represents 
the entire population to which the study's \® Not applicable |= Unable to determine 
indicators apply. 
Comments 
6.4 The instruments for data collection provided for i Met © Partially met lw | Not met 
consistent, accurate data collection over the time : 
periods studied. © Not applicable \® Unable to determine 
Instrument(s) used: 
| | Survey Medical Record Abstraction Tool | | Other 
Comments 
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6.5 The study design prospectively specified a data «| Met 'm| Partially met \m| Not met 
Seana peat m| Not applicable | Unable to determine 
Comments 

6.6 Qualified staff and personnel were used to collect © Met ©) Partially met (=| Not met 

the data. ® Not applicable ‘= Unable to determine 

Name Title 

Role(s) of Project Leader 
Comments 


Step 7: Assess improvement strategies 








7.1 Reasonable interventions were undertaken to ‘m= Met \m| Partially met |=) Not met 
address causes/barriers identified through data 

analysis and QI processes undertaken. ‘© Not applicable ‘= Unable to determine 
Describe Intervention: 

Comments 
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Step 8: Review data analysis and interpretation of study results 





NA if study is not yet complete 


8.1 An analysis of the findings was performed =) Met |m) Partially met =| Not met 
according to data analysis plan. © Not applicable @, Unable to determine 


Not met if study is complete and no indication of a data analysis plan (see step 6.5) 


Comments 


8.2 The MCO/PIHP presented numerical PIP results iS) Met \@ Partially met \@| Not met 


and findings accurately and clearly. 
= y ly = Not applicable \@ Unable to determine 


| | Are tables and figures labeled? |_| Labeled clearly, accurately? 


Comments 


8.3 The analysis identified initial and repeat =| Met ‘| Partially met (=| Not met 
measurements, statistical significance, factors that = : 
influence comparability of initial and repeat 
measurement, and factors that threaten internal and 
external validity. 


| Not applicable \™ Unable to determine 


|=. 


Indicate time periods of 
measurements: 





Indicate statistical analyses 
used: 





Indicate statistical signifi level — ’ 
sinfidenok level eae ee Ree im 99% | 95% '® Unable to determine 





Comments 
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8.4 Analysis of study data included an interpretation — gj) et | Partially met im) Not met 
of the extent to which its PIP was successful and 
follow-up activities. ®) Not applicable \® Unable to determine 


Limitations described: 


Conclusions regarding the success 
of the interpretation: 


Recommendations for follow-up 
Comments 


Step 9: Assess whether improvement is "real" improvement 





Note: NA only if study period is not yet complete: otherwise "Unable to Determine" or "No" 


9.1 The same methodology as the baseline 


m| Met || Partiallymet §|@| Not met 
measurement was used when measurement was , ; 
repeated. =| Not applicable \@ Unable to determine 
Same source of data im| yes '@| No | Not applicable \ Unable to determine 
Same method of data collection = |®! yes =| No =| Not applicable |™ Unable to determine 
Same participants examined ‘m| yes No | Not applicable (= Unable to determine 
Same tools used | yes =| No | Not applicable |\@ Unable to determine 
Comments 
9.2 There was a documented, quantitative im] Met ‘| Partially met lis’ Not met 
improvement in process or outcomes of care. 
‘=| Not applicable |\™| Unable to determine 
\@) increased | decreased 
Statistical significance Clinical significance 
Comments 
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9.3 The reported improvements in performance have © Met '™| Partially met =| Not met 
"face" validity: i.e., the improvement in performance : 
appears to be the result of the planned quality ‘= Not applicable “ Unable to determine 


improvement intervention. 


Degree to which the intervention was i) No relevance i) Small (@| Fair \@ High 
the reason for change: -_ 





Comments 
9.4 There is statistical evidence that any observed iw Met ‘| Partiallymet —{W@) Not met 
performance improvement is true improvement. . 
= Not applicable '™ Unable to determine 
‘=| Weak ®| Moderate lm] Strong 


Comments 





Step 10: Assess sustained improvement 


10.1 Sustained improvement was demonstrated |__| Met | | Partiallymet | | Not met 
through repeated measurements over comparable "] Not applicable | Unable to determine 
time periods. 

Comments 
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ACTIVITY 3: EVALUATE OVERALL VALIDITY AND RELIABILITY OF STUDY 
RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS AND 


RECOMMENDATIONS 
Conclusions 
Recommendations 
Check one: 
\™| High confidence is reported ® Low confidence level is reported in MCO/PIHP PIP results 


\™) Moderate confidence is reported MCO/PIHP PIP results ‘™ Reported MCO/PIHP PIP results not credible 


(| Not Applicable, study not complete 
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Appendix 3 —-PM & PIP Request Documents 


Performance Improvement Project Validation 
General Instructions 


Mail Binder To: 

Attn: External Quality Review Submission 
Behavioral Health Concepts, Inc. 

2716 Forum Blvd., Suite 4a 

Columbia, MO 65203 


Due Date: December 9, 2005 


Please refer to Performance Improvement Project Validation Submission Requirements 
and the MCO Performance Improvement Project Summary. 
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2005 External Quality Review of the MC+ Managed Care Program 


Performance Improvement Project Validation Submission Requirements 


Instructions: The following listing includes relevant source data for the EQR process. Submit paper print outs or 
photocopied items in the supplied EQR 2005 binder using the associated tabs. Please refer to the enclosed 
MCO Performance Improvement Project Summary. Submit information for the two PIPs at your MCO to be 
validated by the EQRO. Place information in the binder behind the associated tab and complete the form 
below. You may also mark PIP sections if desired. Use separate tabs and summary sheets for each PIP. 


lf you have any questions about this request, contact Amy McCurry, EQRO Project Director, amccurry@pmsginfo.com. 


Check submitted Use this field to indicate whether you have submitted this information. If you are not submitting the particular 
information, please indicate “NA”. You may have submitted the content by other means or as part of some other 
documentation. If so, indicate “submitted”, and reference the document (see below). 

Name of Source Please write the name of the document you are submitting for the item. If you are submitting pages from a procedure 


Document manual, indicate so by writing "HEDIS submission manual, pages xx — xx." 

MCO Comments Use this space to write out any concerns you may have or any clarification that addresses any issues or concerns you 
may have regarding either the items requested or what you submitted in the response. 

Reviewed By (BHC use) This space will be for BHC staff use. The purpose will be for tracking what is received and what is not received. It will 
not indicate whether the documents actually address the specific issue. 
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Name of PIP 





MCO Comments Reviewed 
by (BHC 
Cover letter with clarifying information (optional) 


a ee ee 
2. | Project narratives, baseline measures, methods, 
interventions, and planned analyses. Examples of 
information are contained in the CMS protocol, 
Validating of Performance Improvement Projects[ |]. 
We will be looking for the following information in the 
Performance Improvement Project descriptions. 
. Phase-in/timeframe for each phase of each PIP 
. Problem identification 
. Hypotheses 
. Evaluation Questions 
. Description of intervention(s) 
. Methods of sampling, measurement 
. Planned analyses 
. Sample tools, measures, surveys, etc. 
1. Baseline data source and data 


Note: BHC may request raw data files, medical records, or additional data. 


a 
b 
C 
d 
e 
f 
8 
h 
1 
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Performance Measure Validation 
General Instructions 


Mail Binder To: 

Attn: External Quality Review Submission 
Behavioral Health Concepts, Inc. 

2716 Forum Blvd., Suite 4a 

Columbia, MO 65203 


Due Date: December 9, 2005 


When applicable, submit one for each of the three measures: 
e Childhood Immunization Status 
e Well-Child Visits in the First 15 Months of Life 
e Annual Dental Visits 


Unless otherwise indicated, please send all documents in hard copy, using the enclosed 
binder and tabs. If an item is not applicable or not available, please indicate this in the tab. 


General data submission instructions 

Data file formats all need to be ASCII, and readable in a Microsoft Windows environment. 
Use an appropriate delimiter (e.g., @) for data that may contain commas or quotation 
marks. Insure that date fields either contain a null value or a valid date. Files will be 
accepted only in the specified layout. Make all submissions using compact disk (CD) 
formats. Data files submitted via e-mail will not be reviewed. Insure that files on the CD 
are accessible on a Microsoft Windows workstation prior to submitting. 


There should be at least 2 files for each measure: 
|. File |. Enrollment Data; 
2. File 2. Denominator and numerator file; 
3. File 3. For Hybrid Method ONLY; Listing of cases selected for medical record 
review. 


The file layouts to be used for each measure are presented on pages 4-6 of 
this document. 
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Annual Dental Visit 


File |. Enrollment Data 
Please provide all enrollment periods for each eligible MC+ Member to verify continuous 
enrollment and enrollment gaps. 


Field Name 
MCO 

MEASURE 

DCN 

MEMBR_ FIRST 
MEMBR_LAST 
DOB 
ENROLL FIRST 
ENROLL_ LAST 


Content 

MC+ Managed Care Organization name 

ADV = Annual Dental Visits 

The Missouri Medicaid recipient identification number 
MC+ Member First Name 

MC+ Member Last Name 

MC+ Member date of birth 

First date of enrollment 


Last date of enrollment 


File 2. Denominator and Numerator Data 


Field Name 
MCO 
MEASURE 

DCN 
MEMBR_ FIRST 
MEMBR_LAST 
DOB 


STRAT_GROUP 
SER DATE 
SER_CD 


TYPE_CD 


Potonmnisace Banagemont 
Solutions Group 
ee os a a  e. ae 


Content 

MC+ Managed Care Organization name 

ADV = Annual Dental Visits 

The Missouri Medicaid recipient identification number 
MC+ Member First Name 

MC+ Member Last Name 


MC+ Member date of birth 
1=4 - 6-year-olds 

2=7 - 10-year-olds 

3=11 - 14-year-olds 
4=15 - 18- year-olds 
5=19-21-year-olds 


Date of service 


Code used to identify Annual Dental Visit 
Type of coding system: C=CPT Codes; |=|CD-9-CM Codes; 
H=HCPCS/CDT-3 Codes 
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Well-Child Visits in the First 15 Months of Life 


File |. Enrollment Data 
Please provide all enrollment periods for each eligible MC+ Member to verify continuous 
enrollment and enrollment gaps. 


Field Name 
MCO 

MEASURE 

DCN 
MEMBR_ FIRST 
MEMBR_LAST 
DOB 


ENROLL_ FIRST 


ENROLL_LAST 


Content 

MC+ Managed Care Organization name 

W15 = Well-Child Visit in the First 15 Months of Life 
The Missouri Medicaid recipient identification number 
MC+ Member First Name 

MC+ Member Last Name 

MC+ Member date of birth 

First date of enrollment 

Last date of enrollment 


File 2. Denominator and Numerator Data 


Field Name 
MCO 

Measure 

DCN 
MEMBR_ FIRST 
MEMBR_LAST 
DOB 
SER_DATE 
SER_CD 


TYPE CD 
DATA_SOURCE 


HYBRID _EVNT 
ADMIN_EVNT 


NUM_ID 
Note: 


Content 

MC+ Managed Care Organization name 

W15 = Well-Child Visit in the First 15 Months of Life 
The Missouri Medicaid recipient identification number. 
MC+ Member First Name 

MC+ Member Last Name 

MC+ Member date of birth 

Date of service 


Code used to identify numerator event 
Type of coding system: C=CPT Codes; |=ICD-9-CM Codes; M= MCO Internal 
Code* 


For Hybrid Method ONLY 
Please specify source of data: A - Administrative; MR - Medical Record Review 


For Hybrid Method ONLY 
Hybrid numerator event (positive event “hit”): y=yes; n=no 


Administrative numerator event (positive case "hit"): y=yes; n=no 

Seven separate numerators are calculated for this measure. Please indicate if this 
case was counted toward: 

0 = O visits numerator; 1 = 1 visit numerator; 

2 = 2 visits numerator; 3 = 3 visits numerator; 

4 = 4 visits numerator;5 = 5 visits numerator; 

6 = 6 or more visits numerator. 


If MCO used internal codes, please provide codes used and their description in a separate file. 


File 3. For Hybrid method ONLY - please provide a listing of the cases 
selected for medical record review. Use the following layout: 


Field Name 
MCO 

MEASURE 

DCN 
MEMBR_ FIRST 
MEMBR_LAST 
DOB 


MR_STATUS 


Potornmsace Banagemant 
Solutions Group 
— Sheen see coon oo GAC 


Content 

MC+ Managed Care Organization name 

CIS = Childhood Immunization Status 

The Missouri Medicaid recipient identification number 
MC+ Member First Name 

MC+ Member Last Name 


MC+ Member date of birth 
Medical record review status: 
R = reviewed: 
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NR = not reviewed; 
S = substituted 


PROVIDER NAME Primary Care Provider who supplied the record 
PROVIDER_ID Primary Care Provider identification number 


Childhood Immunization Status 

File |. Enrollment Data 

Please provide all enrollment periods for each eligible MC+ Member to verify continuous 
enrollment and enrollment gaps. 


Field Name Content 

MCO MC+ Managed Care Organization name 

MEASURE CIS = Childhood Immunization Status 

DCN The Missouri Medicaid recipient identification number 
MEMBR_ FIRST MC+ Member First Name 

MEMBR_LAST MC+ Member Last Name 

DOB MC+ Member date of birth 

ENROLL_FIRST First date of enrollment 

ENROLL_LAST Last date of enrollment 


File 2. Denominator and Numerator Data 


Field Name Content 

MCO MC+ Managed Care Organization name 

MEASURE CIS = Childhood |mmunization Status 

DCN The Missouri Medicaid recipient identification number. 

MEMBR_ FIRST MC+ Member First Name 

MEMBR_ LAST MC+ Member Last Name 

DOB MC+ Member date of birth 

SER_DATE Date of service 

SER_CD Code used to identify numerator event 

TYPE CD Type of coding system: C=CPT Codes; |=ICD-9-CM Codes. 
For Hybrid Method ONLY 

DATA_SOURCE Please specify source of data: A - Administrative; MR - Medical Record Review 
For Hybrid Method ONLY 

HYBRID _EVNT Hybrid numerator event (positive event “hit”): y=yes; n=no 

ADMIN. EVNT Administrative numerator event (positive case "hit"): y=yes; n=no 

EXCLUD Was the case excluded from denominator Y=Yes; N=No 

EXCLUD_REASON Reason for exclusion 


File 3. For Hybrid method ONLY - please provide a listing of the cases 
selected for medical record review. Use the following layout: 


Field Name Content 
MCO MC+ Managed Care Organization name 
MEASURE CIS = Childhood Immunization Status 
DCN The Missouri Medicaid recipient identification number 
MEMBR_ FIRST MC+ Member First Name 
MEMBR_LAST MC+ Member Last Name 
DOB MC+ Member date of birth 
rhea Performance Management Solutions Group 
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Medical record review status: 
MR_STATUS R = reviewed; NR = not reviewed; S = substituted 
PROVIDER NAME Primary Care Provider who supplied the record 
PROVIDER_ID Primary Care Provider identification number 


Please see the Performance Measure Validation Submission Requirements and the Summary 
of Calculation Methods for Performance Measures. 


Casex Performance Management Solutions Group 
nee A division of Behavioral Health Concepts, Inc. 


483 


Missouri MC+ Managed Care External Quality Review Appendix 3 
Report of Findings — 2005 PM & PIP Request Documents 


2005 External Quality Review of the MC+ Managed Care Program 


Performance Measure Validation Submission Requirements 


Instructions: The following listing includes relevant source data for the EQR process. Submit paper print outs or photocopied items in the 
supplied EQR 2005 binder using the associated tabs. Within each tab, include information specific for each of the three measures for the 
MC+ population. Some items may not apply. For example, if you do not use a HEDIS vendor and perform measure calculations on site, 
then you may not have documentation on electronic record transmissions. These items apply to processes, personnel, procedures, 
databases and documentation relevant to how the MCO complies with HEDIS measure calculation, submission and reporting. 


If you have any questions about this request, contact Amy McCurry, EQRO Project Director, amccurry@pmsginfo.com. 


Check submitted Use this field to indicate whether you have submitted this information. If you are not submitting the particular 
information, please indicate “NA”. You may have submitted the content by other means either on the BAT or as part of 
some other documentation. If so, indicate “submitted”, and reference the document (see below). 

Name of Source Please write the name of the document you are submitting for the item. If you are submitting pages from a procedure 


Document manual, indicate so by writing "HEDIS submission manual, pages xx — xx." 

MCO Comments Use this space to write out any concerns you may have or any clarification that addresses any issues or concerns you 
may have regarding either the items requested or what you submitted in the response. 

Reviewed By (BHC use) This space will be for BHC staff use. The purpose will be for tracking what is received and what is not received. It will 
not indicate whether the documents actually address the specific issue. 
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Tab | HEDIS Performance Measure Check if | Name of Source MCO Comments 
Submitted Document 
or NA 


HEDIS 2005 Data Submission Tool (MO DHSS 2005 Table 
B HEDIS Data Submission Tool) for all three measures for 
the MC+ Managed Care Population only. Do not include 
other measures or populations. 

2005 HEDIS Audit Report. This is the HEDIS 
Performance Audit Report for the MC+ Managed Care 
Program product line and the three MC+ measures to be 
validated (complete report). If the three measures to be 
validated were not audited or if they were not audited for 
the MC+ Managed Care Program population, please send 
the report, as it contains Information Systems Capability 
Assessment information that can be used as part of the 
Protocol. 

Baseline Assessment Tool (BAT) for HEDIS 2005. The 
information submitted for the BAT will include 
descriptions of the process for calculating measures for 
the MC+ Managed Care Program population. 


List of cases for denominator with all HEDIS 2005 data 
elements specified in the measures. 


Performance Mame gon 
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List of cases for numerators with all HEDIS 2005 data 
elements specified in the measures, including fields for 
claims data and MOHSAIC, or other administrative data 
used. Please note that one of the review elements in the 
Protocol is: The “MCO/PIHP has retained copies of files 
or databases used for performance measure reporting, in 
the event that results need to be reproduced.” 

List of cases for which medical records were reviewed, 
with all HEDIS 2005 data elements specified in the 
measures. Based on a random sample, BHC will request 
MCOs to gather a maximum of 30 records per measure 
and submit copies of the records requested to BHC. 


Sample medical record tools used if hybrid method(s) 
were utilized for HEDIS 2005 Well-Child Visits or 
Childhood Immunization Status measures for the MC+ 
Managed Care Program population; and instructions for 
reviewers. 


All worksheets, memos, minutes, documentation, policies 
and communications within the MCO and with HEDIS 
auditors regarding the calculation of the selected 
measures. 
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Check if | Name of Source MCO Comments 
Submitted Document 
or NA 
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Tab | HEDIS Performance Measure Check if | Name of Source MCO Comments Reviewed 
Submitted | Document by (BHC 
use) 


Policies, procedures, data and information used to 
produce numerators and denominators. 


Policies, procedures, and data used to implement sampling 
(if sampling was used). At a minimum, this should include 
documentation to facilitate evaluation of: 

a. Statistical testing of results and any corrections 
or adjustments made after processing. 

b. Description of sampling techniques and 
documentation that assures the reviewer that 
samples used for baseline and repeat 
measurements of the performance measures 
were chosen using the same sampling frame 
and methodology. 

c. Documentation of calculation for changes in 
performance from previous periods (if 
comparisons were made), including tests of 
Statistical significance. 





Policies and procedures for mapping non-standard codes. 


Record and file formats and descriptions for entry, 
intermediate, and repository files. 





= ae 
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13. | Electronic transmission procedures documentation. (This 
will apply if the MCO sends or receives data electronically 
from vendors performing the HEDIS abstractions, 


Check if | Name of Source 





calculations or data entry.) 


Descriptive documentation for data entry, transfer, and 
manipulation of programs and processes. 


Documentation of proper run controls and of staff review 
of report runs. 


15. | Samples of data from repository and transaction files to 
assess accuracy and completeness of the transfer process. 
7. 


| Documentation of results of statistical tests and any 
corrections or adjustments to data along with justification 
for such corrections or adjustments. 


Ors Performance Management Solutions Group 
teen BE A division of Behavioral Health Concepts, Inc. 








Appendix 3 
PM & PIP Request Documents 


MCO Comments Reviewed 
by (BHC 
use) 





488 


Missouri MC+ Managed Care External Quality Review 
Report of Findings — 2005 


2. 


HEDIS Performance Measure Check if 
Submitted 





Documentation of sources of any supporting external data 
or prior years’ data used in reporting. 


Procedures to identify, track, and link member enrollment 
by product line, product, geographic area, age, sex, 
member months, and member years. 


Procedures to track individual members through 
enrollment, disenrollment, and possible re-enrollment. 


Procedures used to link member months to member age. 


Documentation of “frozen” or archived files from which 
the samples were drawn, and if applicable, documentation 
of the MCO’s/PIHP’s process to re-draw a sample or 
obtain necessary replacements. 
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Tab | HEDIS Performance Measure Check if | Name of Source MCO Comments Reviewed 
Submitted | Document by (BHC 
or NA 





Procedures to capture data that may reside outside the 
MCO’s/PIHP’s data sets (e.g. MOHSAIC). 


24. | Policies, procedures, and materials that evidence proper 
training, supervision, and adequate tools for medical 
record abstraction tasks. (May include training material, 
checks of inter-rater reliability, etc.) 
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Performance Measures to be Calculated for MC+ Members 

METHOD FOR CALCULATING HEDIS 2005 PERFORMANCE MEASURES 
Please complete this form and place in the HEDIS 2005 section of the binder supplied by BHC. Please direct 
any questions to Amy McCurry or Mariya Chumak. 
MCO 
Date Completed 
Contact Person 
Phone 


Fax 


NCQA Accredited for MC+ Product (Yes/ No) 


Certified HEDIS Software Vendor and Software 


Record Abstraction Vendor 


‘@lalitelaterere| Well-Child Visits 


Mostaskeralr+-iacelal in the First 15 Annual Dental 
Measure to be validated by EQRO | Status Months of Life | Visits 





What was the reporting Date for HEDIS 2005 
Measures ? 


What was the Audit Designation (Report/ No 
Report/ Not Applicable) ? 


Was the measure publicly Reported (Yes/ No) ? 
Did denominator include members who 
switched MCOs (Yes/ No) ? 

Did denominator include members who 


switched product lines (Yes/ No) ? 


Did the denominator include 1115 Waiver 
Members (Yes/ No) ? 


Were proprietary or other codes (HCPC, NDC) 
used ? 





Administrative 


Were exclusions calculated (Yes/ No) ? 


Hybrid 





On what date was the sample drawn ? 
Were exclusions calculated (Yes/ No) ? 
How many medical records were requested ? 


How many medical records were received ? 
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APPENDIX 4 
PERFORMANCE MEASURES 


WORKSHEETS 
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Appendix 4 — Performance Measures Worksheets 


Final Performance Measure Validation Worksheet: HEDIS 2005 Well-Child Visits in the First 15 
Months of Life 


The percentage of enrolled members who turned 15 months during the measurement year, who received either 
zero, one, two ,three, four, five, six or more well-child visits with a primary care practitioner during their first 
15 months of life. 


Element Specifications Comments 
Documentation 


Appropriate and complete measurement plans and programming 
specifications exist that include data sources, programming logic, and 
computer source code. 

Eligible Population 
Age year. 
31 days - 15 months of age. Calculate the 15-month 
Enrollment birthday as the child's first birthday plus 90 days. 


No more than one gap in enrollment of up to 45 days 
during the continuous enrollment period. To determine 
continuous enrollment for an MC+ beneficiary for whom 
enrollment is verified monthly, the member may not have 
more than a 1-month gap in coverage. 

Enrolled on the day the child tuns 15 months old 

Medical 


























Event/diagnosis |None 
Sampling 


Sampling was unbiased. 
Sample treated all measures independently. 
Sample size and replacement methods met specifications. 
Numerator 
Data sources used to calculate the numerator (e.g., member ID, claims 








files, medical records, provider files, pharmacy records, including those for 
members who received the services outside the MCOs network) are 
complete and accurate. 

Calculation of the performance measure adhered to the specification for all 
components of the numerator of the performance measure. 

Documentation tools used were adequate. 

Integration of administrative and medical record data was adequate. 

The results of the medical record review validation substantiate the reported 
numerator. 














Denominator 
Data sources used to calculate the denominator (e.g., claims files, medical 
records, provider files, pharmacy records) were complete and accurate. 
X=) elelaa tare 


State specifications for reporting performance measures were followed. 


Estimate of Bias 
What range 0 - 5 percentage points 
defines the > 5 - 10 percentage points 
impact of data |> 10 - 20 percentage points 
incompleteness |> 20 - 40 percentage points 
for this > 40 percentage points 
measure? Unable to determine 


What is the 
direction of the 
bias? Overreporting 

















Underreporting 




















Audit Rating 
Fully Compliant = Measure was fully compliant with State specifications. 


Substantially Compliant = Measure was substantially compliant with State specifications and had only minor 
deviations that did not significantly bias the reported rate. 


Not Valid = Measure deviated from State specification such that the reported rate was significantly biased. This 
designation is also assigned to measures for which no rate was reported, although reporting of the rate was 
required. 


Not Applicable = No MC+ Members qualified 


Note: 2 = Met; 0 = Not Met 
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Final Performance Measure Validation Worksheet: HEDIS 2005 Childhood Immunization 
Status 


The percentage of enrolled children two years of age who had four DTaP/DT, Three IPV, one MMR, 
three H influenza type B, three hepatitis Ba nd one chicken pox vaccine (VAB) by the time period 
specified and by their second birthday. The measure aslo calculates two separate combination rates. 


Element Specifications Rating Comments 


Appropriate and complete measurement plans and programming 
specifications exist that include data sources, programming logic, and 
computer source code. 

Eligible Population 
Children who turn two years of age during 





Age the measurement year. 
Twelve months prior to the child's second 
Enrollment birthday. 





No more than one gap in enrollment of up 
to 45 days during the 12 months prior to 
the child's second birthday. To determine 
continuous enrollment for a MC+ member 
for whom enrollment is verified monthly, 
the member may not have more than a 1- 
month gap in coverage during the 12 











Gap months prior to the child's second birthday 
Anchor date Enrolled on the child's second birthday 
Benefit Medical. 

Event/diagnosis None. 








Sampling 


Sampling was unbiased. 
Sample treated all measures independently. 

















Sample size and replacement methods met specifications. 






Numerator 


Data sources used to calculate the numerator (e.g., member ID, 
claims files, medical records, provider files, pharmacy records, 
including those for members who received the services outside the 
MCOs network) are complete and accurate. 





Calculation of the performance measure adhered to the specification 
for all components of the numerator of the performance measure. 
Denominator 
Data sources used to calculate the denominator (e.g., claims files, 
medical records, provider files, pharmacy records) were complete and 











accurate. 

X=) lel adiale, 
State specifications for reporting performance measures were 
followed. 


Estimate of Bias 
0 - 5 percentage points 
> 5 - 10 percentage points 
What range defines the |> 10 - 20 percentage points 

















impact of data > 20 - 40 percentage points 
incompleteness for this > 40 percentage points 
measure? Unable to determine 


Underreporting 





What is the direction of 
the bias? Overreporting 

Audit Rating 
Fully Compliant = Measure was fully compliant with State specifications. 

















Substantially Compliant = Measure was substantially compliant with State specifications and had only minor 
deviations that did not significantly bias the reported rate. 


Not Valid = Measure deviated from State specification such that the reported rate was significantly biased. This 
designation is also assigned to measures for which no rate was reported, although reporting of the rate was 
required. 


Not Applicable = No MC+ Members qualified 


Note: 2 = Met; 0 = Not Met 
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Final Performance Measure Validation Worksheet: HEDIS 2005 Annual Dental Visit 


The percentage of enrolled MC+ Managed Care Program Members who were 3 -21 years of age who had 
at least one dental visit during the measurement year. This measure applies only if dental care is a 
covered benefit in the MCO's Medicaid contract. 


Element Specifications Rating Comments 
Documentation 


Appropriate and complete measurement plans and programming 
specifications exist that include data sources, programming logic, 
and computer source code. 

Eligible Population 


4 -21 years of age as of December 31, 
2004. The measure is reported for 
each of the following age 
Stratifications and as a combined rate: 
* 4 -6 year-olds 

* 7-10 year-olds 

* 11 - 14 year-olds 

* 15 - 18 year-olds 

* 


Age 19 - 21 year-olds 





Enrollment Continuous during 2004 





No more than one gap in enrollment 
of up to 45 days during 2004. To 
determine continuous enrollment for 
an MC+ beneficiary for whom 
enrollment is verified monthly, the 
member may not have more than a 1- 
Gap month gap in coverage. 

Anchor date Enrolled as of December 31, 2004 
Benefit Medical 















Event/diagnosis None 
Sampling - Not Applicable to this measure, calculated via Administrative calculation 














Numerator 


Data sources used to calculate the numerator (e.g., member ID, 
claims files, medical records, provider files, pharmacy records, 
including those for members who received the services outside the 
MCOs network) are complete and accurate. 





Calculation of the performance measure adhered to the specification 
for all components of the numerator of the performance measure. 


Denominator 






Data sources used to calculate the denominator (e.g., claims files, 
medical records, provider files, pharmacy records) were complete 
and accurate. 





























X=) ole] aa lare| 
State specifications for reporting performance measures were 
followed. 
Estimate of Bias 
0 - 5 percentage points 
> 5 - 10 percentage points 
What range defines the > 10 - 20 percentage points 
impact of data > 20 - 40 percentage points 
incompleteness for this > 40 percentage points 
measure? Unable to determine 
Underreporting 
What is the direction of the 
bias? Overreporting 








PANU Colima thadtale, 
Fully Compliant = Measure was fully compliant with State specifications. 





Substantially Compliant = Measure was substantially compliant with State specifications and had only 
minor deviations that did not significantly bias the reported rate. 


Not Valid = Measure deviated from State specification such that the reported rate was significantly 
biased. This designation is also assigned to measures for which no rate was reported, although reporting 
of the rate was required. 


Not Applicable = No MC+ Members qualified 


Note: 2 = Met; 0 = Not Met 
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APPENDIX 5 
ENCOUNTER DATA 


MINIMUM CRITERIA 
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Appendix 5 — Encounter Data Minimum Criteria 


Recommended Encounter Data Validation Criteria 
Data Element Expectation WEV Ife Tia a Og ik-jal-| 


Enrollee ID Should be valid as 100% valid 
found in the State's 
eligibility file. 


Principal Diagnosis Well-coded lead- > 90% non-missing and valid 
related diagnoses codes. 
(or well-child visit) 

Date of Service Dates should be If looking at a full year of data 
evenly distributed 5-7% of the records should be 
across time distributed across each month. 


Unit of Service (Quantity) The number should 98% non-zero 
be routinely coded. <70% should be one if CTP 
code in range of 99200-99215, 
99241-99291 


Procedure Code This is a critical 99% present (not zero, blank, 
element and should ~ 8- or 9-filled). 100% should be 
always be coded. valid, State-approved codes. 
Will be assessed There should be a wide range of 
only for presence of | procedures with the same 
code except for frequency as previously 
lead-related codes encountered. 
which will be 
validated with 
medical records. 


Source: Medstat (1999). A Guide for States to Assist in the Collection and Analysis of 
Medicaid Managed Care Data:: Second Edition 
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APPENDIX 6 
ENCOUNTER DATA 


REQUEST LETTER 
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Appendix 6 — Encounter Data Request Letter 


Behavioral Health Concepts, Inc. 
Victoria Park, 2716 Forum Blvd., Suite 4, Columbia, MO 65203 (573) 446-0405 

(573) 446-1816 (fax) 

(866) 463-6242 (toll-free) 

www.bhcinfo.com 





December 22, 2005 
Re: 2005 External Quality Review Encounter Data Validation Protocol 
Dear < >, 


As discussed with MCO staff at the All-Plan meeting and during the 2005 EQR orientation 
meeting over teleconference, BHC is requesting the following information for Encounter Data 
Validation from each MCO: 


|. File 1: Mailing address and contact information of provider associated 
with each Internal Control Number (ICN) for the sampled claims 
(service dates January |, 2005 to March 31, 2005). BHC requires this 


information for tracking purposes. Due date: January 6, 2006. 


2. File 2: All inpatient encounters from January |, 2005 to March 31, 2005 
for the selected recipients (MC+ members associated with each sampled 
claim). 


3. File 3: All outpatient encounters (Outpatient, Medical, Dental, and Home 
Health) from January |, 2005 to March 31, 2005 for the selected 
recipients (MC+ members associated with each sampled claim). 


4. File 4: All pharmacy encounters from January |, 2005 to March 31, 2005 
for the selected recipients (MC+ members associated with each sampled 
claim). 


Due date for file 2, 3, and 4 is February 10 2006. 


Enclosed is a CD-ROM containing a file of the sample of encounters. This file contains claim 
ICNs (Internal Control Number) and patient identifying information. Please use this sample to 
request medical records from providers. 


We recognize that it is a busy holiday season, and are allowing up to seven business days for 
preparation of the medical record requests. The requests must be submitted to providers by 
January 6", 2006. This will allow the providers 4 weeks to gather records. Providers should 
supply records directly to BHC, Inc. by February 3, 2006. MC+ Managed Care Organizations 
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are extended an additional week to submit records you collect from providers. Records not 
received by February 10", 2006 will be considered undocumented encounters. Please be 
advised that BHC and/or DMS do not provide reimbursement for the cost of photocopying or 
mailing records. 


During the past three years BHC provided a status report to MCOs indicating the submission 
rate of records during the collection process. This practice is intended to facilitate a higher 
return rate. In order to provide this service, BHC must obtain requested provider information. 
Please return provider contact information to BHC, in the requested format, by January 6", 
2006. 


To assist with the medical record request process, we have also enclosed medical records 
submission instructions, a letter from Sandra Levels, and information regarding federal and state 
requirements for adherence to HIPAA and the External Quality Review. 


If you have any questions, please contact BHC’s External Quality Review team at 573-446-0405. 
We would like to take this opportunity to wish you very Happy Holidays! 


Thank you, 


Amy McCurry, Esq. 
EQRO Project Director 


1. Encounter Validation Submission Instructions 

2. File layouts for submission of inpatient, outpatient and pharmacy data 
3. Medical Records Submission Instruction 

4. Letter from Sandra Levels 

5. CD-ROM with sample of encounters for encounter data validation 

6 


CC: 
Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Department of Social 
Services, Division of Medical Services 
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Encounter Data Validation Submission Instructions 


Mail To: 


Behavioral Health Concepts, Inc. 
Attn: Mariya Chumak 

2716 Forum Bivd., Suite 4a 
Columbia, MO 65203 


Label the package CONFIDENTIAL 
Due Dates: 


Provider information file — January 6", 2006; 

Inpatient, Outpatient, Pharmacy encounter data — February 10, 2006. 
Provider must submit medical records to BHC, Inc. — February 3, 2005 
MC+ MCOQOs are extended an additional week to submit medical records 
they collected from providers — February 10, 2006 


hAWN = 


General data submission instructions 

All data file formats must be ASCII, and readable in Microsoft Windows environment. Use 
an appropriate delimiter (e.g., @) for data that may contain commas or quotation marks. 
Insure that date fields either contain a null value or a valid date. Make all submissions using 
compact disk (CD) formats and mail it to BHC, Inc. No files will be accepted via e-mail. 
Insure that files on the CD are accessible on a Microsoft Windows workstation prior to 
submitting. 


Specific data submission instructions 


EncounterDataRequest.xls. file contains six worksheets. 


|. The first worksheet, "Read Me", provides detailed instructions for responding to this 
data request. 

2. The second worksheet contains a sample of claims selected for Encounter Data 
Validation for your Health Plan. 


Please use the listing of ICNs in "Data to BHC" and generate a report that contains the 
following information for the sample: 


PROVIDER FIRST NAME 
PROVIDER LAST NAME 
PROVIDER TITLE 
PROVIDER ADDRESS | 
PROVIDER ADDRESS 2 
PROVIDER CITY 
PROVIDER STATE 
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PROVIDER ZIP CODE 


Return the ENTIRE worksheet "Data to BHC" to BHC, Inc. by January 6*, 2006 with 
detailed provider information. 


Please use this sample to request medical records from providers. The requests must be 
submitted to providers by January 6, 2006. This will allow the providers 4 weeks to gather 
records. Providers should supply records directly to BHC, Inc. by February 3, 2006. MC+ 
Managed Care Organizations are extended an additional week to submit records you collect 
from providers. Records not received by February 10", 2006 will be considered 
undocumented encounters. 


3. The third worksheet, "Sample", contains a sample of DCN numbers selected to conduct 
comparison between State encounter claims and MCO encounter claims. Please provide all 
inpatient, outpatient, and pharmacy encounter claims for the selected DCNs for service 
dates from January |, 2005 to March 31, 2005. This information must be received by 
February 10th, 2006. 


Files will be accepted ONLY in layouts provided in worksheets 4, 5 and 6. 


4. The worksheet "Inpatient" contains the file layout to be used for submission of inpatient 
encounter data. 

5. The worksheet "Outpatient" contains the file layout to be used for submission of 
outpatient encounter data. 

6. The worksheet "Pharmacy" contains the file layout to be used for submission of 
pharmacy encounter data. 
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Encounter Data Request File Layout 


File layout for submission of inpatient claims. 


Field Name Content Description 
ICLAIM TYPE Claim type: 
| = Inpatient 


ICLAIM STATUS P=Paid 
U=Unpaid 
D=Denied 


[ICN 
= Health Plan Claim Internal Control Number 


IPAID) AMT 
— This field indicates the amount of money paid to the hospital for the billed 
services. 


IRECIP ID i, , ee ces 
— The Missouri Medicaid recipient identification number. 
|LAST -_ 
Recipient last name 
IFIRST _ 
Recipient first name 


|IACCT NUM The recipient's account number used by the doctor's office. 


[ADMIT TYPE 


Admission Type 
The only valid values are: 
1 = Emergency 
2 = Urgent 
3 = Elective 
4 = Newborn 


9 = Information Not Available 


|IADM_ DT 
— The date the recipient was admitted to the hospital. This date cannot exceed 
the current date. 
IDSCH_ DT 
7 The date the recipient was discharged from the hospital. If the patient is still 
in the hospital, the latest date of service that applies to the claim. 
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IBILL_ TYPE 


Valid bill type codes are: 


Inpatient 

11x 

12x 

18x 
Outpatient 

13x 

14x 

71x (Rural Health) 

81x (Hospice) 

82x (Hospice) 
Home Health 

30x 

31x 

32x 

33x 

34X 

35x 

36x 

37x 


38x 


39x 
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ISTAT 


The code that represents the condition under which the recipient was 
discharged. 


O01 Home 

O2 Hospital 

03 Skilled Nursing Facility (SNF) 

04 Intermediate Care Facility (1 CF) 
O05 Institution (Inst) 

O06 Home Health Agency (HHA) 

O07 Left 

08 Other 

20 Death 

30 Still A Patient 

50 Discharge from Hospice to Home 
51 Discharge from Hospice to Another Medical Facility 


62 Discharged/transferred to an inpatient rehabilitation facility (IRF) 
including rehabilitation distinct part units of a hospital 


64 Discharged/transferred to a nursing facility certified under Medicaid 
but not certified under Medicare 


65 Discharged/transferred to a psychiatric hospital or psychiatric 
distinct part unit of a hospital 


IPROV. NUM a . 
~ The Health Plan’s 9-digit provider number. 


IPRIM DX , . 
= The recipient's primary diagnosis. Decimal points are implied. 
IDX 2 . . 
a Second diagnosis. Decimal points are implied. 
IDX 3 
— Third diagnosis. Decimal points are implied 





IDX_4 

= Fourth diagnosis. Decimal points are implied. 
IDX_5 

— Fifth diagnosis. Decimal points are implied. 


IKEY A code that indicates the patient has other insurance that may or may not be 
reflected on the claim. Valid values are: 
1 = Yes, patient has other insurance. 


2 = Yes, patient has other insurance not reflected on this bill. 
3 = No, patient does not have other insurance. 


IFDT SVC 
a The date that the billing period begins. 
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ILDT SVC 
The date that the billing period ends. 


IREVENUE CD 3 
~ The three-digit code from 100 to 999 that represents the services that are 


billed on this particular line item. The combined total number of 
accommodation and ancillary services billed cannot exceed 28 lines per claim. 


Accommodation revenue codes range from 10X through 21X. Ancillary 


revenue codes range from 22X through 99X. 


NOTE: Emergency Room (rev 450 and 459) and Ambulance (rev 540 to 549) 


may only be billed as inpatient if the patient is admitted to the hospital. 


IUNITS SVC 
— The number of days per room rate for both covered and non-covered 


accommodations (revenue codes 100 through 239). Whole numbers only are 
accepted for the days. 
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File layout for submission of outpatient claims. 


Field Name Content Description 


OCLAIM_TYPE O=Outpatient 
M=Medical 
L=Dental 
H=Home Health 

OCLAIM STATUS Claim Type: O, M, L, H 
P=Paid 
U=Unpaid 
D=Denied 


This field is informational only and reflects what FFS would pay. 
ORECIP ID 
~ Claim Type: O, M, L, H 
es The Missouri Medicaid recipient identification number. 
OLAST 
Claim Type: O, M, L, H 
ee Recipient last name 
OFIRST 
Claim Type: O, M, L, H 
oe Recipient first name 
OACCT NUM Claim Type: O, M, L, H 
The recipient's account number used by the doctor's office. This 
ee field may be left blank or used for other purposes, such as the 
Health Plan Claim Internal Control Number. 
OPROV NUM 
~ Claim Type: O, M, L, H 
ee The Health Plan’s 9 digit provider number. 
mibaam 
The |CD-9 diagnosis code of the recipient's diagnosis. Any 
decimal point needed in the diagnosis code Is implied and should 
not be included. 
st 
Second diagnosis. The |CD-9 diagnosis code of the recipient's 
diagnosis. Any decimal point needed in the diagnosis code Is 
implied and should not be included. 
carn 
Third diagnosis. The |CD-9 diagnosis code of the recipient's 
diagnosis. Any decimal point needed in the diagnosis code is 
implied and should not be included. 
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ODX_4 
ODX_5 


O KEY 


OFIRSTDT_SVC 
OLASTDT_SVC 





Encounter Data Request Letter 


Claim Type: O, M, L, H 


Fourth diagnosis. The |CD-9 diagnosis code of the recipient's 
diagnosis. Any decimal point needed in the diagnosis code is 
implied and should not be included. 

Claim Type: O, M, L, H 


Fifth diagnosis. The |CD-9 diagnosis code of the recipient's 
diagnosis. Any decimal point needed in the diagnosis code is 
implied and should not be included. 

Claim Type: O, M, L, H 


A code that indicates the patient has other insurance that may or 
may not be reflected on the claim. Valid values are: 

O = No, patient does not have other insurance. 

1 = Yes, patient has other insurance. 

2 = Yes, patient has other insurance not reflected on this Dill. 
Claim Type: O, M, L, H 

This is the first date the service was performed. This date cannot 
exceed the current date. 

Claim Type: O, M, L, H 

This is the last date the service was performed. This date cannot 
exceed the current date. 
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OPLACE SVC Claim Type: M, L 


C-14 PLACE OF SERVICE 

03 School 

O04 Homeless Shelter 

05 Indian Health Service Free-Standing Facility 
06 Indian Health Service Provider-Based Facility 
07 Tribal 638 Free-Standing Facility 

08 Tribal 638 Provider-Based Facility 

11 Office 

12 Home 

13 Assisted Living Facility 

14 Group Home 

15 Mobile Unit 

20 Urgent Care Facility 

21 Inpatient Hospital 

22 Outpatient Hospital 

23 Emergency Room - Hospital 

24 Ambulatory Surgical Center 

25 Birthing Center 

26 Military Treatment Facility 

31 Skilled Nursing Facility 

32 Nursing Facility 

33 Custodial Care Facility 

34 Hospice 

41 Ambulance - Land 

42 Ambulance - Air or Water 

49 Independent Clinic 

50 Federally Qualified Health Center (FQHC) 

51 Inpatient Psychiatric Facility 

52 Psychiatric Facility - Partial Hospitalization 
53 Community Mental Health Center 

54 Intermediate Care Facility/Mentally Retarded 
55 Residence Substance Abuse Treatment Facility 
56 Psychiatric Residential Treatment Facility 

57 Non-Residential Substance Abuse Treatment Facility 
60 Mass Immunization Center 

61 Comprehensive Inpatient Rehabilitation Facility 
62 Comprehensive Outpatient Rehabilitation Facility 
65 End Stage Renal Disease Treatment Facility 
71 State or Local Public Health Clinic 

72 Rural Health Clinic 

81 Independent Laboratory 

97 Parochial/Private Schools 

98 Schools 

99 Other Unlisted Facility 


Claim Type: O, H 
Not applicable 


OUTPAT UNITS SVC Claim Type: O, M, L, H 
The number of units of services performed. Whole numbers only. 
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ODTL PROC Claim Type: M,L, H 
The procedure code that represents the service preformed. 
Claim Type: O 
For outpatient claims, a procedure code is required only when 
the revenue code range for outpatient services is 300 through 
319. This revenue code range represents laboratory services. 
The appropriate CPT procedure code range for laboratory 
services is 80048 through 89399. All other outpatient services 
must be designated by revenue code. 


The 2-digit modifier that applies to the service provided. 
The 2-digit modifier that applies to the service provided. 


ODTL_DIAG CODE Claim Type: O, M, L, H 
The diagnosis code of the recipient's diagnosis. Decimal points 
are implied. 


OREVENUE_CD Claim Type: O 
The three digit code from 100 to 999 which represents the 
services that are billed on this particular line item. A revenue 
code Is required on all Outpatient claims. For those revenue 
codes representing lab services (300-319), a procedure code 
must also be submitted. 
Claim Type: M, L, H 
Not applicable 
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File layout for submission of Pharmacy Claims. 


Field Name Content Description 


PH TRANSACTION CD This field shows the number of claims being billed on the 
record. Valid values are: 


01 - 1 Claim 


O02 - 2 Claims 
03 - 3 Claims 
04 - 4 Claims (maximum) 


PHCLAIM STATUS P=Paid 
U=Unpaid 
D=Denied 


PHICN Health Plan Claim Internal Control Number 
PH PROV NUM The Health Plan’s 9-digit provider number 


PH NABP_NUM This field will always contain the 7-digit National Association 
of Boards of Pharmacy (NABP) identification number assigned 
to the pharmacy. The NABP number must be in the first 7 
positions of the 9-digit field (left justified). 


PHRECIP ID The Missouri Medicaid recipient identification number. 


PHKEY A code that indicates the patient has other insurance that 
may or may not be reflected on the claim. Valid values are: 
O = No, patient does not have other insurance. 
1 = Yes, patient has other insurance. 
2 = Yes, patient has other insurance not reflected on this Dill. 


PH FIRST DT SVC The dispense date. 


PH LAST 
— Entire name may be entered. Only the first two letters of the 
recipient's last name and the first letter of the recipient's first 
name will be verified against the recipient's Medicaid 
enrollment records. The plan must send a minimum of two 
characters for the last name and one character for the first 
name. 
PH FIRST 
— Entire name may be entered. Only the first two letters of the 
recipient's last name and the first letter of the recipient's first 
name will be verified against the recipient's Medicaid 
enrollment records. The plan must send a minimum of two 
characters for the last name and one character for the first 
name. 


PHREFILL_IND The only valid values are: 
Original - OO (zero) 
Refill - 01-99 
dispensed. For example: A quantity of 100 would be 0100. 
represents. A days supply greater than 365 is invalid. 
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PHCOMPOUND_IND An indicator identifying the prescription as a non-compound 
or as an ingredient of a compound prescription. 
A value of '0' or '1' is used to indicate non-compound 
prescriptions or the FIRST ingredient of a compound 
prescription. A value of '2' is used to indicate any additional 
ingredients of a compound prescription. 


PHARM DRUG NDC CODE The National Drug Code designated for the drug dispensed. 
The field is 5-4-2 format no hyphens or spaces 


PHPROV_ NUM The Medicaid, DEA number, or name of the prescribing 
physician. If not available, enter the dispensing pharmacy 
NABP number unless you are a pharmacy having FQHC 
Status. 


PHEPSDT_IND A code indicating whether or not a drug was dispensed to a 
recipient under the Early Periodic Screening and Diagnostic 
Treatment (EPSDT) program. Y = yes 
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Medical Record Submission Instructions 


As discussed with MCO staff in the 2005 EQR orientation meeting over 
teleconference, this year MCOs will be requesting medical record for 
encounter data validation. The CD submitted with this request contains a 
Microsoft Excel file with 100 sample encounters. Please match each encounter 
with a provider that substantiates a claim and request them to supply medical 
records to BHC, Inc. We are interested in all services provided to these 
patients by the designated provider from January |, 2005 through March 31, 
2005. This information is used to document the volume and type of services 
provided to MC+ Managed Care Program Members and to validate the 
accuracy and completeness of the State encounter claims database. 


For each medical record please request the following: 

e Face/Demographic sheet or other documentation that identifies the patient receiving 
services and primary care provider, January |, 2005 through March 31, 2005. This 
information includes: 

o Patient Name 
o Medicaid ID Number 
o Date of Birth 
o Provider Name 
o Provider Number 
e Documentation of all services (professional, physician’s/doctor’s 
orders, laboratory test results) from January |, 2005 through March 
31, 2005. Sources for this information may include: 
o Primary Diagnosis 
Progress Notes 
Laboratory findings 
Treatment Plans 
Claim Forms or Superbills 
Flow Sheets 


OOO 0 0 


Behavioral Health Claims: 

Due to the sensitive nature of these records, please instruct your providers to 
submit only the primary diagnosis code, claim information, and minimum necessary 
information to support the diagnosis and procedure codes for which services were 
billed during the specified time frame (January |, 2005 — March 31, 2005). Providers 
must not send raw test data, protocols, or shadow charts. Additionally, recommend 
your behavioral health providers to de-identify names of individuals related to the 
patient in progress notes. 


Providers should include any and all information to support the 
procedures for which a claim was submitted. 


Records not received by the due date will be considered 
undocumented encounters 
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Please note that providers should submit records directly to BHC by February 3, 
2005 


Please note that BHC will not reimburse providers or copy services 
for copy and postage costs. Please encourage providers not to submit 
invoices. 


In the interest of confidentiality, please DO NOT FAX or E-MAIL any 


forms or portions of medical records. 


Records should be mailed to: 


Attn: Mariya Chumak 
Behavioral Health Concepts, Inc. 
2716 Forum Bivd. Ste. 4A 
Columbia, MO 65203 


Please label the package “CONFIDENTIAL” 
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Behavioral Health Concepts, Inc. 


Victoria Park, 2716 Forum Blvd., Suite 4, Columbia, MO 65203 (573) 446-0405 
(573) 446-1816 (fax) 
(866) 463-6242 (toll-free) 
www.bhcinfo.com 





® 


January 6, 2005 


Subject: 2005 External Quality Review Performance Measure Validation 
Protocol Medical Records Request (hybrid methodology only). 


Due Date: February 10, 2006 


Dear < >, 


We have reviewed < > for the HEDIS 2005 Childhood 
Immunization Measure and Well-Child Visits in the First 15 Months of Life Measure. 


Please find enclosed a CD-ROM containing a file with a listing of cases selected for 
medical record review. We are requesting copies of medical records for the sampled 
cases that contributed to the numerator. Please forward copies of the medical records 
to Behavioral Health Concepts, Inc. (BHC) at the address listed above, and mark the 
package as confidential. 


If you have any questions, please contact BHC’s EQRO team at (573) 446-0405 or via e- 
mail: eqro@bhcinfo.com 


Thank you, 
Amy McCurry 
EQRO Project Director 


Encl.: 
|) CD with a sample of cases for medical record review 


cc: Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri 
Department of Social Services, Division of Medical Services 
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Well-Child (WI5) Abstraction Tool 


Patient Name 


Date of Birth 
Missing = 99999999 


Provider Name 


First 


Name of MCO L} Community Care Plus (1) LJ Family Health Partners (5) 
(Check only one) LJ Mercy Health Plan (2) LJ FirstGuard (6) 
LJ HealthCare USA (3) LJ Blue Advantage Plus (7) 
LJ Missouri Care (4) 


Abstractor 
Initials 


Date of 
abstraction 


Data entry 
operator initials 
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Search the medical record for a well care visit during the calendar year 


Source of 
Documentation: LJ Medical Record (1) 
LJ Claim Form (2) 
LJ Both (3) 
LJ None (0) 
Documented Health and Developmental History 
Components LJ Yes (1) 
of Well Care LJ No (0) 
Visit: 
(Check all that Physical Exam 
apply) LJ Yes (1) 
LJ No (0) 
Anticipatory Guidance 
LJ Yes (1) 
LJ No (0) 
Date of Well Care 
Visit 1 m m d d y 


Unless ALL 
components above 


are checked, code 
Missing = 99999999 


Missing = 99999 
Insufficient Information = 22222 
Don't Know = 88888 
See list to the right of Procedure Codes. Does procedure code match one of 
these? 
Procedure Code 
Match 1 LY Yes (1) 
Q No (0) 


Diagnosis Code 1 
Decimal is implied. 


Start at left. If only 3 
or 4 digits, leave the 
right spaces blank. 
Missing = 99999 


Insufficient 
Information = 22222 


Don't Know = 88888 


Diagnosis Code LO Yes (1) 
Match 1 LY No (0) 


Acceptable Procedure Codes: 


99381 99382 99391 99392 


99432 


Acceptable Diagnosis Codes: 
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Date of Well Care 

Visit 2 mM mM d d y y y y 
Unless ALL 

components above 

are checked, code 

Missing = 99999999 


Acceptable Procedure Codes: 


99381 99382 99391 99392 


Missing = 99999 99432 
Insufficient Information = 22222 

Don't Know = 88888 

See list to the right of Procedure Codes. Does procedure code match one of 





these? Acceptable Diagnosis Codes: 
Procedure Code 
Match 2 LY Yes (1) 

O8 No (0) 





Diagnosis Code 2 
Decimal is implied. 


Start at left. If only 3 
or 4 digits, leave the 
Missing = 99999 
Insufficient 


Information = 22222 
Don't Know = 88888 


Diagnosis Code UO) Yes (1) 

Match 2 LY No (0) 
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Date of Well Care 
Visit 3 

Unless ALL 
components above 


are checked, code 
Missing = 99999999 


Procedure Code 3 


Missing = 99999 


Insufficient Information = 22222 


Don't Know = 88888 


Table of Contents for Med Rec. Training Manual 
Abstraction Tools 


Acceptable Procedure Codes: 


| | | ft [| 99381 99382 99391 99392 


99432 





See list to the right of Procedure Codes. Does procedure code match one of 


these? 
Procedure Code 
Match 3 


Diagnosis Code 3 
Decimal is implied. 


Start at left. If only 3 


or 4 digits, leave the 
right spaces blank. 
Missing = 99999 
Insufficient 
Information = 22222 


Don't Know = 88888 


Diagnosis Code 
Match 3 


Date of Well Care 
Visit 4 

Unless ALL 
components above 


are checked, code 
Missing = 99999999 


Procedure Code 4 


Missing = 99999 


Insufficient Information = 22222 


Don't Know = 88888 


Acceptable Diagnosis Codes: 


OW Yes (1) 
CY No (0) 





| | | | 


OW Yes (1) 
CY No (0) 


Acceptable Procedure Codes: 


| | | ft [| 99381 99382 99391 99392 


99432 





See list to the right of Procedure Codes. Does procedure code match one of 


these? 
Procedure Code 
Match 4 


Diagnosis Code 4 
Decimal is implied. 


Start at left. If only 3 


or 4 digits, leave the 
right spaces blank. 
Missing = 99999 
Insufficient 
Information = 22222 


Don't Know = 88888 


Diagnosis Code 
Match 4 


Salutlens Group 
oe ol eon ae ore on AC 


Gea Performance Management Solutions Group 


Acceptable Diagnosis Codes: 


OW Yes (1) 
LY No (0) 





| | | | 


OW Yes (1) 
L) No (0) 
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Date of Well Care 

Visit 5 m m d d y y y y 
Unless ALL components 

above are checked, 

code Missing = 

99999999 


99381 99382 99391 99392 


Missing = 99999 99432 
Insufficient Information = 22222 
Don't Know = 88888 


See list to the right of Procedure Codes. Does procedure code match one of these? Acceptable Diagnosis Codes: 
Procedure Code 
Match 5 O) Yes (1) 

O No (0) 


Diagnosis Code 5 
Decimal is implied. 
Start at left. If only 3 
or 4 digits, leave the 
right spaces blank. 
Missing = 99999 
Insufficient Information 
= 22222 


Don't Know = 88888 


Diagnosis Code 
Match 5 


Date of Well Care 
Visit 6 

Unless ALL components 
above are checked, 
code Missing = 
99999999 


Acceptable Procedure Codes: 


Procedure Code 6 
99381 99382 99391 99392 


Missing = 99999 99432 


Insufficient Information = 22222 
Don't Know = 88888 


See list to the right of Procedure Codes. Does procedure code match one of these? Acceptable Diagnosis Codes: 
Procedure Code 
Match 6 O) Yes (1) 

O) No (0) 


Diagnosis Code 6 
Decimal is implied. 


Start at left. If only 3 
or 4 digits, leave the 
right spaces blank. 
Missing = 99999 


Insufficient Information 
= 22222 


Don't Know = 88888 


Diagnosis Code 
Match 6 
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Childhood Immunization Abstraction Tool 


Patient Name 


Date of Birth: 
Missing = 99999999 


Provider Name 


First 
Name of MCO L} Community Care Plus (1) LJ Family Health Partners (5) 
(Check only one) LJ Mercy Health Plan (2) LJ FirstGuard (6) 
L) HealthCare USA (3) LJ Blue Advantage Plus (7) 
LJ Missouri Care (4) 


Abstractor I nitials 
Date of abstraction 


Data entry operator 
initials 


Start Time 
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Search the medical record for the complete immunization history 


Source of Documentation: 
Check One 


Medical Record (1) 
Claim Form (2) 
Both (3) 

None (0) 


Type of Documentation 
Check One 


Dated Immunization History (1) 
Immunization Certificate (2) 
Both (3) 

None (0) 


DUOU UUUO 


Is There Evidence of a History of: 
Measles Yes (1) 
No (0) 


Mumps Yes (1) 
No (0) 


Rubella Yes (1) 
No (0) 


Measles Seropositive Test Date 
Missing = 99999999 
Not Applicable = 88888888 


Mumps Seropositive Test Date 
Missing = 99999999 
Not Applicable = 88888888 


Rubella Seropositive Test Date 
Missing = 99999999 
Not Applicable = 88888888 


MMR Date 1 
Missing = 99999999 
Not Applicable = 88888888 


First Birthday 


42 days old 


Second Birthday 
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Was one of the MMRs completed after the member's first birthday and before the LJ Yes (1) 
member's second birthday? LJ No (0) 





Source of Documentation: LJ Medical Record (1) 

(Check all that apply) LJ Claim Form (2) 

Type of Documentation: LJ} Dated Immunization History (1) 
(Check only one) LJ Immunization Certificate (2) 


Is there documented evidence of a history of Hep B? L) Yes (1) 
LJ No (0) 


Hep B Seropositive Test Result Date 
Missing = 99999999 
Not Applicable = 88888888 


Hep B Date 1 
Missing = 99999999 
Not Applicable = 88888888 


At delivery/birth = 11111111 


Hep B Date 2 
Missing = 99999999 


Not Applicable = 88888888 


Hep B Date 3 
Missing = 99999999 


Not Applicable = 88888888 


Wasa Hep B completed between 6 months of age and the member's first birthday? LJ Yes (1) 
LJ No (0) 
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DTa/DT 


Source of Documentation: Medical Record (1) 
Check One Claim Form (2) 
Both (3) 
None (0) 


Type of Documentation Dated Immunization History (1) 
Check One Immunization Certificate (2) 
Both (3) 
None (0) 


Is There Documented Evidence of a History of: 
Diptheria LJ Yes (1) 
LJ No (0) 


Tetanus LJ Yes (1) 
LJ No (0) 


DTaP/DT Date 1 
Missing = 99999999 
Not Applicable = 88888888 


DTaP/DT Date 2 
Missing = 99999999 
Not Applicable = 88888888 


DTaP/DT Date 3 
Missing = 99999999 
Not Applicable = 88888888 





Notes: 
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OPV/IPV 


Source of 
Documentation: 


Check One 


Medical Record (1) 
Claim Form (2) 
Both (3) 

None (0) 


Type of Documentation 
Check One 


Dated Immunization History (1) 
Immunization Certificate (2) 
Both (3) 

None (0) 


DOOOUOUWU OOOD 


Is There Documented Evidence of a History 

of: 
OPV seropositive Test 
date 


|IPV seropositive Test 
date 


\ 

OPV/IPV Date 1 
Missing = 99999999 

Not Applicable = 88888888 


OPV/IPV Date 2 
Missing = 99999999 
Not Applicable = 88888888 


OPV/IPV Date 3 
Missing = 99999999 
Not Applicable = 88888888 
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Medical Record (1) 
Claim Form (2) 
Both (3) 

None (0) 


Source of Documentation: 
Check One 


Type of Documentation 


Dated Immunization History (1) 
Immunization Certificate 

(2) 

Both (3) 


None (0) 


Check One 


OOO O OUUL 


HiB Date 1 
Missing = 99999999 
Not Applicable = 88888888 


HiB Date 2 
Missing = 99999999 
Not Applicable = 88888888 


HIB Date 3 
Missing = 99999999 
Not Applicable = 88888888 


HiB Date 4 
Missing = 99999999 
Not Applicable = 88888888 
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VZV 


Source of Documentation: L} Medical Record (1) 
Check One L} Claim Form (2) 
LJ) Both (3) 
L} None (0) 
Type of Documentation LJ =Dated Immunization History (1) 
Immunization Certificate 
Check One LI (2) 
LJ) Both (3) 
L} None (0) 
Yes 
Is There Documented Evidence of a History of Chicken Pox? LJ (1) 
No 
L} (0) 


Date of positive Chicken Pox? mM mM d d y y 
Not Applicable = 88888888 

VZV Date 1 m m d d 

Not Applicable = 88888888 
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Medical Record Abstraction Tool 


ICN Primary Key 

Patient Name OUTPAT_RECIP_LAST_NAME OUTPAT_RECIP_FIRST_NAME 
Date of Birth OUTPAT_RECIP_BIRTHDATE 

Patient DCN OUTPAT_PROCESSED_RECIP_ID 

Provider Name FIELD 

Clinic Name FIELD 

Clinic Address 

First Date of Service FIELD 

Abstractor I nitials | 


Data entry operator initials | 


Examine only the information provided in physician and professional documentation. DO NOT 
use the CMS-1500, any claim forms, or any claim histories. 


Medical Record 


Error 
Comparison Type 


OUTPAT FIRST DT SVC 


Date of Service 


Comment (Required if Error Type = Other) 


Primary Diagnosis OUTPAT DX_1 


Decimal is implied. Start at left. If only 3 or 4 digits, leave the right spaces blank. 


Comment (Required if Error Type = Other) 


DX_ DESCRIPTION 
Primary Diagnosis 
Description 
Comment (Add description from medical record; Required if Error Type = Other) 
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Patient Name OUTPAT_RECIP_LAST_NAME OUTPAT_RECIP_ FIRST NAME 
Date of Birth OUTPAT_RECIP_ BIRTHDATE 
Patient DCN OUTPAT_ PROCESSED RECIP_ID 


Element | ode 
Procedure Code To be coded by reviewer 


Decimal is implied. Start at left. If only 3 or 4 digits, leave the right spaces blank. 


Not Enough Information = 22222 ef | ff 


Comment (Required if Error Type = Other) 


To be coded by reviewer 


Procedure Description 


Comment (Add description from medical record; Required if Error Type = Other) 


Referrals Documented in the Medical Record (check all that apply; only if not related to 
the claim validated) 


None (0) 
Laboratory (1) 
Pharmacy (2) 
Specialist (3) 
Radiology (4) 
Other (5) 





See next page for the procedure code and procedure code description to be validated. 
Does the medical record documentation adequately support the procedure code and 
description? 

LJ Yes (1) 

LJ No (0) 

If no, Reason (check only one): 

Not enough information (e.g., the date of service and 
information are present, but there is not enough 
information to make a determination) (1) 
Upcoded (2) 
Incorrect (3) 
Missing (9) 
Other (4) 


DOOOOU 


Comment 
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Patient Name OUTPAT_RECIP_LAST_NAME OUTPAT_RECIP_FIRST_NAME 
Date of Birth OUTPAT_RECIP_ BIRTHDATE 
Patient DCN OUTPAT_PROCESSED RECIP_ID 


Examine the CMS-1500 or any claim forms. If there is no claim form or history, code as missing. 


Claim Form or History 


Error 
Comparison Type 


OUTPAT FIRST DT SVC ace 
= —— 0 = No Jonly 1, 
: 1 = Yes i 9, or 
pees LIL tele et Mats I ae We Ms He | 


Comment ( (Required if Error Type = Other) 


Primary Diagnosis OUTPAT_DX_1 ode 
ae 0 = No Jonly 1, 
1 = Yes]3, 8, 9 


Decimal is implied. Start at left. If only 3 or 4 digits, leave the right spaces blank. 
Missing = 99999 tt ft ft 


Comment (Required if Error Type = Other) 


DX_DESCRI PTION 


Primary Diagnosis 


Description 
Comment (Required if Error Type = Other) 


OUTPAT DTL PROC 


Procedure Code 
Comment (Required if Error Type = Other) 


OUPT DESCR 


Procedure Description 
Comment (Required if Error Type = Other) 
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Appendix 8 — Agenda for Site Visits 






Together we can chart the course, 
measure accomplishments, and 
quantify the outcomes. 





Performance Management 
Solutions Group | 
a Division of Behavioral Health Concepts, Inc. BHC. 





February 21, 2006 


RE: SITE VISIT AGENDA AND DOCUMENT REVIEW 


We are finalizing plans for the on-site reviews of each MCO. We are providing the following 
information in an effort to make preparation for the on-site review as efficient as possible for 
you and your staff. The following is information or persons needed at the time of the on-site 
review at (MCO name). 


Performance Improvement Projects 


Time is scheduled in the afternoon to conduct follow-up questions, review databases, and 
provide verbal feedback to the MCO regarding the planning, implementation, and credibility 
of findings from the Performance Improvement Projects (PIPs). Any staff responsible for 
planning, conducting, and interpreting the finings of PIPs should be present during this time. 
The review will be limited to the projects and findings submitted at the end of 2005. Please 
be prepared to review databases and any data collection forms not originally submitted. 


Performance Measure Validation 
As you know, BHC 1s in the process of validating the following three performance measures: 


# HEDIS 2005 Annual Dental Visits 
# HEDIS 2005 Childhood Immunization Status, Combination #2 
»# HEDIS 2005 Well-Child Visits in the First Fifteen Months of Life 


BHC is following the CMS protocol for validating performance measures. The goals for this 
process are to: 


= Evaluate the accuracy of Medicaid performance measure reported by the MCO; and 

= Determine the extent to which Medicaid-specific performance measures calculated by the 
MCO followed specifications established by the Division of Medical Services. These 
specifications consist of the HEDIS 2005 Technical Specifications. 
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To complete this process we will review the following documents while on-site: 


« Data Integration and Processes Used to Calculate and Report Performance 
Measures 


Documentation of the performance measure generating process 

Report production logs and run controls 

3. Documentation of computer queries, programming logic, or source code (if available) 
used to create denominators, numerators and interim data files - for each of the three 
measures 

4. Code mapping documentation 

5. Documentation of results of statistical tests and any corrections with justification for such 
changes, if applicable - for each of the three measures 

6. Documentation showing confidence intervals of calculations when sampling methodology 
used — for each of the three measures 

7. Description of the software specifications or programming languages instructions used to 
query each database to identify the denominator, and/or software manual 

8. Source code for identifying the eligible population and continuous enrollment calculation 
— for each of the three measures 

9. Description of the software specification or programming languages used to identify the 
numerator 

10. Programming logic and/or source code for arithmetic calculation of each measure to 

ensure adequate matching and linkage among different types of data 


NO — 


= Sampling Validation 


Description of software used to execute sampling sort of population files 

Source code for how samples for hybrid measures were calculated 

3. Policies to maintain files from which the samples are drawn in order to keep population 
intact in the event that a sample must be re-drawn or replacements made 

4. Documentation that the computer source code or logic matches the specifications set forth 

for each performance measure, including sample size and exclusion methodology 

Documentation of “frozen” or archived files from which the samples were drawn 

6. Documentation assuring that sampling methodology treats all measures independently, 

and there is no correlation between drawn samples 


NO —_— 


- 


Performance Measure Interviews 


In addition to the documentation reviews, interviews will be conducted with the person(s) 
responsible for: 
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= Overseeing the process of identifying eligible members from MCO data sources for the 
measures to be validated; 

= Programming the extraction of required elements from the MCO data sources for the 
measures to be validated; 

= Integrity checks and processes of verifying the accuracy of data elements for the measures 
to be validated; 

= Overseeing the process of medical record abstraction, training, and data collection for the 
measures to be validated; and 

= Contractor oversight and management of any of the above activities. 


On-site activities may also include, but are not limited to, the following: 


# Demonstration of HEDIS software 
= Demonstration of the process for extracting data from MCO databases 
= Possible data runs for identifying numerator and denominator cases 


Compliance Review 


The final activity to prepare for during the on-site visit will be the compliance follow-up 
review. Documentation review and interviews with Division of Medical Services staff have 
occurred prior to the on-site visit. This will enable BHC to use the time at the MCO as 
efficiently as possible. The following information will be needed at the time of the on-site 
review: 


Compliance Documents 


= Member Handbook 

= Provider Handbook 

# Provider Agreements 

#" Marketing Plan and materials 

= Policies and procedures requested on-site 


Attached 1s a listing of grievance and appeal records, for both members and providers, which 
will be reviewed during the on-site visit. We are requesting that you have these records 
available when BHC arrives on-site. 


Compliance Interviews 


The agenda requests an interview in the morning with the leadership from the MCO. It would 
be helpful to include the following staff: 


# Plan Director 
"= Medical Director 
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*" Quality Assurance Director 
# Provider Services/Provider Relations Director 
= Member Services Director 
"= Utilization Management Director 


Interviews are scheduled in the afternoon to discuss mental health services. We are 
requesting that staff from subcontractors be available during this time period. 


There are concurrent activities and interviews scheduled in the morning and the afternoon. If 
separate conference rooms or meeting space can be arranged, this will make the process much 
easier to coordinate. Also, the on-site review team will need to order a working lunch on the 
day of the visit. If lunch facilities are not available, please provide the name and telephone 
number of a service in the vicinity to accommodate ordering lunch. Your assistance will be 
appreciated. 


The MCO staff involved in any of the referenced interviews or activities, or anyone identified 
by the MCO, is welcome to attend the introduction or the exit interview. 


Your assistance in organizing the documents, individuals to be interviewed, and the day’s 
activities is appreciated. If you have questions, or need additional information, please let me 
know. 


Sincerely, 


Mona Prater 
Assistant Project director 


Cc: Amy McCurry, Esq., Project Director 
Plan Administrator 
Judy Muck, Division of Medical Services 


Attachment: 
On-Site Review Agenda 
Grievance and Appeal Case Review Listing 
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APPENDIX 9 
COMPLIANCE REVIEW 


SCORING FORM 
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Appendix 9 — Compliance Review Scoring Form 


2005 BHC MCO Compliance Review Scoring Form 


This document is used to score the number of items met for each regulation by the MCO. 
|. Review all available documents prior to the site visit. 
2. Follow-up on incomplete items during the site visit. 


3. Use this form and the findings of Interviews and all completed protocols to complete the Documentation and Reporting Tool and rate 
the extent to which each regulation is met, partially met, or not met. 


Scores from this form will be used to compare document compliance across all MCOs. 

0 = Not Met: Compliance with federal regulations could not be validated. 

| = Partially Met: MCO practice or documentation indicating compliance was observed, but total compliance could not be validated. 

2 = Met: Documentation is complete, and on-site review produced evidence that MCO practice met the standard of compliance with federal 


regulations. 


PAU Oho My i x=) 2004 Rating PAU OMe ldlate 
WiAtiiare late 0 =Not Met UR Cola a (-15 


ola te-les Federal 2004 Site Visit | Findings | = Partially Met | = Partially Met 
Compliance Tool Regulation DY-Yol a felatela Comments FeValom i tatel tate f 2 = Met 2 = Met 


Subpart C: Enrollee Rights and Protections 
1(a)1-25, Enrollee Rights: 
6(a), 2.6.2(j) 438.100(a) General Rule 
1(a)1, 2.9, Enrollee Rights: Basic 
2(j), 2.6.2(n) 438.10(b) Rule 
Alternative Language: 
3 2.15.2(e), 2.8.2 438.10(c)(3) Prevalent Languages 


2.6. 
D2: 
2.6. 
2.6. 
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Compliance Review Scoring Form 


| 2004 Rating 
0 =Not Met 
| = Partially Met 


5 Rating 
ot Met 
artially Met 


Visit and 
a takel tate (3 





| Compliance Tool 


2 One. O. oy 
4 2.6.2(n)(2) 


438.10(c)(4,5) 


Description 
Language and format: 
Interpreter Services 
Information 


2.6.1(a)1, Requirements: 
5 2.6.2(n)1 438.10(d)(1) (i) Alternative Formats 
2.6.1(a)1, 
2.6.2(n)2 - dot Information 
point 35, 2.6.2(q), 438.10(d)(1)(ii)an Requirements: Easily 
6 2.8.2, 2.8.3 d (2) Understood 
2.3.5, 2.6.1(a)2/3, 
2.6.2(k)1, 2.6.2(n), Enrollee Rights: 
2.6.2(n)(2), Information, Free 
7 2.6.2(q) 438.10(f) Choice 
Information to 
Enrollees: Physician 
8 2.6.2(n)(2) 438.10 (g) Incentive Plans 
2.4, 2.4.5, 
2.4.5(a)2-4, Liability for Payment 
9 2.20.1(all), 3.5.3(f) | 438.10(i) and Cost Sharing 
Specific Enrollee 
2.2.6(a), 2.2.6(b), Rights: Provider- 
2.6.1(a)(3), Enrollee 
10 | 2.6.2(j), 2.9.1 438.100(b)(2) (iii) Communications 


2.6.2(j), 2.30.1, 


438.100(b)(2)(iv,v 


Right to Services, 
including right of 
refusal. Advance 


ll 2.30.2, 2.30.3 ) Directives 
2.6.2(j), 2.4.8, 

12 | 2.13, 2.14 438.100(b)(3) Right to Services 
2.2.6, 2.14.3, Compliance with Other 


13) 2.14.8, 2.14.9 


438.100(d) 


State Requirements 


| 2 = Met 





Subpart D: Quality Assessment and Performance I mprovement 
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2005 Site | 2004 Rating 2005 Rating 
Wittianelate 0 =Not Met 0 =Not Met 
Contract Federal 2004 Site Visit | Findings | = Partially Met | = Partially Met 
Compliance Tool Regulation Description Comments and Findings 2 = Met 2 = Met 
2.3.1, 2.6.2(j), Availability of 
2.14.3, 2.7.1(g), Services: Provider 
14 3.5.3 438.206(b)(1)(i-v) Network 


2.7 Ae), 2.7. 1\Xt), 


Access to Well 
Woman Care: Direct 


15 | 2.14.8 438.206(b)(2) Access 
16 2.13 438.206(b)(3) Second Opinions 
2.3.2) 2.3.18, Out of Network 
2.7.1(bb), 2.12.3, Services: Adequate 
17 | 2.12.4, 2.14.5 438.206(b)(4) and Timely Coverage 
Out of Network 
Providers: Cost 
18 2.4, 2.20.1(d) 438.206(b)(5) Sharing 
2.3.14(a)2, 2.14.1, 
2.14.4(a-f), 2.17.1, 
JES) |e rene) 438.206(c)(1)(i-vi) Timely Access 
Cultural 
20 2.2.6(a)1-3, 2.17.1 438.206(c)(2) Considerations 


Primary Care and 
Coordination of 


21 2.14.11, 2.3.5(e) 438.208(b) Healthcare Services 
2.6.2(m), 2.14.11, Care Coordination: 
22 2.5.3(e) 438.208(c)(1) |dentification 
2, 12,110), 2,4. 2G), 
2.14.11, 2.17.5, 
Attachment 3 - 
Children with Special Care Coordination: 
23 Healthcare Needs 438.208(c)(2) Assessment 
Care Coordination: 
24 2.7.1, 2.12, 2.14.11 438.208(c)(3) Treatment Plans 
2.3.8, 2.3.7, 
2.6.1(k)(3), 2.14.6, 
25 | 2.14.7 438.208(c)(4) Access to Specialists 
2.2.1(i), 2.3.7, 
2.7.4, 2.9.2, 2.10.2, 
2.14.1, 2.14.2(a-h), Authorization of 
26 2.14.2(d)1-2 438.210(b) Services 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 


550 


Missouri MC+ Managed Care External Quality Review Appendix 9 
Report of Findings — 2005 Compliance Review Scoring Form 





2005 Site | 2004 Rating | 2005 Rating 
Wittianelate 0 =Not Met 0 =Not Met 
Contract Federal PA travel fale [= | = Partially Met | = Partially Met 
Compliance Tool Regulation Description Comments and Findings 2 = Met 2 = Met 
Notice of Adverse 
27 2.15.4, 2.14.2(d)6 438.210(c) Action 
2.6.2(k)(3), 
2.14.2(d)6, 
2.15.4(a-c), Timeframe for 
28 2.16.3(e) 438.210(d) Decisions 
Compensation for 
Utilization 
Management 
29 2.17.5(b) 438.210(e) Decisions 
2.4.8, 2.7.1, Emergency and Pos- 
2.7.1(y), 2.7.3(v), Stabilization pgs 
30) 2.14.2 438.114 24/25 Rev. Checklist 
Subpart D: Quality Assessment and Performance Improvement: Structure and Operation Standards 
General Rules for 
2.17.2(n), 2.17.5(c), Credentialing and 
31 2.30.2 438.214(a,b) Recredentialing 
Nondiscrimination 
and Provider 
438.214(c) and Discrimination 
32 2.2.6(b)(c) 438.12 Prohibited 
32° 122310 438.214(d) Excluded Providers 
Other State 
Requirements: 
34 | 2.3.9, 2.3.17 438.214(e) Provider Selection 
2.6.2(n)(2), Disenrollment: 
2.6.2(s) (all), 438.226 and Requirements and 
35 2.6.2(u) 438.56(b)(1-3) Limitations 
Disenrollment 
2p 2 See 2150) Requested by 
36) §=6.2.6.1(g), 2.6.2® 438.56(c) Enrollee 


Procedures for 
Disenrollment -- Pgs 
37) -2.6.2(r,s-1,t) 438.56(d) 29/30 Rev. Checklist 
Timeframe for 
Disenrollment 


38 = =2.6.2(u) — 438.56(e) - Determinations 
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| 2005 Rating 
0 =Not Met 
| = Partially Met 
| 2=Met 


2004 Rating 
0 =Not Met 

| = Partially Met 
2 = Met 


| 2005 Site 
Visit and 
2004 Site Visit | Findings 


Vale Mm mlateliate i 


lm =\e( =) =] | 
Regulation 


(@elahag-les 


fexoy any elit-lale-me mele) Comments 





Description 


39 


40 


41 


42 


43 


44 


45 


46 


47 


48 


49 


50 


15, 2.15.3(a,b) 
6.1(a)(18), 
16.2(c), 
Bi2(alye, 2313, 
5 


2: 
2. 
2. 
2. 
Shey Fens m mene ne. 


438.228 


438.230(a,b) 


Grievance Systems 


Subcontractual 
Relationships and 
Delegation 


Subpart D: Quality Assessment and Performance I mprovement: Measurement and Improvement 


2.17.2(d) 


2.17.2(d) 


2.17.2(d,f) 


2A A 2 eS 
2.17.5(d) 

Celie lS. 
Attachment 6 
2.17.5(b) 

2175 

Attachment 6 - State 
Quality Strategy 


2.25 


438.236(b)(1-4) 


438.236(c) 


438.236(d) 


438.240(a)(1) 
438.240(b)(1) and 
438.240(d) 
438.240(b)(2)(c) 
and 438.204(c) 


438.240(b)(3) 
438.240(b)(4) 
438.240(e) 


438.242(a) 


Adoption of Practice 
Guidelines 
Dissemination of 
Practice Guidelines 
Application of 
Practice Guidelines -- 
Pgs 32/33 of Rev. 
Checklist 

Quality Assessment 
and Improvement 
Program 

Basic Elements of 
MCO QI and PIPs 
Performance 
Measurement 

Basic elements of 
MCO QI and PIPs: 
Monitoring Utilization 
Basic elements of 
MCO QI and PIPs 
Program Review by 
State 

Health Information 
Systems 
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| 2005 Rating 
0 =Not Met 


PAO elatalte 
0 =Not Met 

| = Partially Met 
2 = Met 


2005 Site 

Visit and 

| = Partially Met 
| 2= Met 


a ake tate (3 





Description 
2.25(all) - 2.25.1, 


2.25.2(a,b), 2.25.3, Basic Elements of 


51 2.25.4 438.242(b)(1,2) HIS 
Basic Elements of 
52 | 2.26.1, 2.29.1 438.242(b)(3) HIS 





Total Quality Improvement and 
Assessment 


Subpart F: Grievance Systems 

Grievance and 

Appeals: General 
Requirements 

Grievance and 

Appeals: Filing 

Authority 

Grievance and 


53 


2.15 438.402(a) 
2eld.2 021 0525 (al); 
Dual 


54 438.402(b)(1) 


55 2.15.6(a) 438.402(b)(2) Appeals: Timing 
2.15.2(a), 2.15.5(a), Grievance and 
56 2.15.6(a,b) 438.402(b)(3) Appeals: Procedures 
Notice of Action: 
2.15.2(e), Language and 
57 2.15.4(a),2.6.2(q) 438.404(a) Format 
Notice of Action: 
58 | 2.15.4(b) 438.404(b) Content 
Notice of Action: 
59 2.15.4(c) 438.404(c) Timing 
Handling of 
Grievances and 
Zr De, Gl), Appeals: General 
60 2.15.6(h,1,j) 438.406(a) Requirements 
Handling of 
Grievances and 
2.15.6(g) 2.15.6(h) Appeals: Special 
61 2.15.6(i) 2.15.6(j) 438.406(b) Requirements 
Resolution and 
notification: 
Grievances and 
62 2.15.5(e), 2.15.6(k) 438.408(a) Appeals - Basic rule 
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FeVale Mm tialelialte t= 


lm =\e(=) =] | 
| Regulation 


(oliiae- leas 
| Compliance Tool | Description Comments 
| Resolution and 

notification: 
Grievances and 
Appeals - 
Timeframes and 
438.408(b,c) extensions 
Resolution and 
notification: 
Grievances and 
Appeals - Format and 
content 
Resolution and 
notification: 
Grievances and 
Appeals - 
Requirements for 
State fair hearing 
Expedited resolution 
of appeals 
Information about 
the grievance 
systems of providers 
and subcontractors 
Recordkeeping and 
reporting 
Continuation of 
Benefits while the 
MCO/PIHP Appeal 
and the State Fair 
69 = 2.15.6(p) 4388.420 Hearing are Pending 
Effectuation of 
— 2.15(a,r) 438.424 reversed appeals 


63 


NN 


64 


438.408(d)(e) 


65 2.15.2(i), 2.15.6(m) | 438.408(f) 


438.410 


66 2.15.6(n,o) 


67 | 2.15.2(c), 3.5.3(c) 438.414 


2.15.3 438.416 


70 
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Appendix 10 -MCO Comments on Draft Report 
BlueAdvantage Plus of Kansas City 


From: J udy Brennan [mailto:] udy.Brennan@BCBSKC.com] 

Sent: Monday, May 15, 2006 4:40 PM 

To: Amy McCurry 

Cc: Darren Taylor; Carrie Cowdin; Michelle Williams; Barb Purdon 
Subject: RE: Performance Measure DRAFT documents 


We have no comments. Thanks. 


Judy Brennan 

Director, State Programs 

BCBSKC 

816-395-2421 * FAX: 816-802-4437 


judy.brennan@bcbskc.com 


From: Amy McCurry [mailto:amccurry@pmsginfo.com] 
Sent: Wednesday, May 10, 2006 3:04 PM 
Subject: Performance Measure DRAFT documents 


Hello All — 


BHC has sent Performance Measure DRAFT documents to you for plan review. You should 
receive them via Fed-Ex today. 


Your packets should include copies of the following: 
@ Performance Measure Objectives and Technical Methods 
¢ Final Performance Measure Validation Worksheets for each 
performance measure (Draft) 

¢ Validation of Performance Measure summary for your MCO (Draft) 
The due date for Plan comments is May 22” — this is a receipt date — all comments must 
be received by May 22, 2006 in order to give BHC time to review and include them in 
the final report if necessary. 


If you have any questions or need further clarification, as always, please don’t hesitate to 
contact me. 


Thank you. 


Amy B. McCurry, Esq., MHSA 
EQRO Project Director 


Performance Management Solutions Group 
a division of Behavioral Health Concepts, Inc. 


2/16 Forum Blvd, Suite 3A 
Columbia, MO 65203 
(573)446-0405 
(573)446-1816 (fax) 
amccurr msqinfo.com 
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Children’s Mercy Family Health Partners 


Children’s Mercy Family Health Partners 
Draft review 5/13-5/15/2006 
Response to draft of Validation of Performance Measures, EQRO 2005 


Pages 3 and 4, Processes Used to Produce Denominators 


Well-Child Visits measure 

‘*A total of 1,677 eligible members were reported and 1,454 eligible members were 
validated for the Well-Child Visits measure.” ‘““The denominators reported for the 
Well-Child Visits measure were over-reported by 223. Of the 223 members the 
EQRO excluded from the denominator, 7 were excluded because their date of birth 
did not fall between October 1, 2002 and September 30, 2003, the other 216 were 
excluded because they were not enrolled in the health plan on the anchor date of 
their 15-month birthday.” 


The NCQA specification for the measure indicates that the 15-month birthday should be 
calculated by adding 90 days to the first birthday. Using this logic to determine the range 
of birthdays that would result in a 15-month birthday anchor date during the 2005 
measurement year, the correct range should be October 3, 2002 through October 2, 
2003. All of the members submitted had birthdates within this range, eliminating the 7 
exclusions due to date of birth. 


The other 216 exclusions need further explanation from BHC because the data we sent 
them shows that the 15-month anchor date does fall within a valid enrollment period. 


Annual Dental Visit measure 

*24,342 eligible members were reported and 24,334 were validated for the 
denominator of the Annual Dental Visit measure.” ‘“The denominators reported for 
the Annual dental Visit measure were over-reported by 8 members. Of the 8 
members that the EQRO excluded from the denominator, 7 were excluded because 
they were not enrolled on the anchor date of December 31, 2004 and one was 
excluded because the member had a gap of enrollment greater than 45 days during 
the measurement year.” 


The denominator for the Annual Dental Visit measure was prepared on April 28, 2005. 
The data for the EQRO audit was prepared on November 28, 2005. Eligibility data for 
the denominator members was extracted from the health plan’s production system 
eligibility file on that date. In reviewing the draft report, 1t was discovered that 
retroactive eligibility terminations and changes occurred after the denominator was 
prepared, and that the production eligibility file records only the changed eligibility dates. 
However, all work files from the denominator processing were archived, and these files 
show the eligibility dates that were in production at the time the denominator was 
processed, and that all members were eligible on the December 31 anchor date. In 


tegoraatenen Performance Management Solutions Group 
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addition, there is an eligibility history file in the production system that shows when the 
changes were made, and the before and after values of the changes, confirming that 
eligibility of these members was valid for the measure. 


Tables of the archived denominator data and eligibility history for the 7 members whose 
eligibility end date are in question can be provided. 


I was not able to identify the member who had a gap of more than 45 days in eligibility 
from our copy of the data. 


Page 4, Processes Used to Produce Numerators — Childhood Immunizations Status 


measure 
‘Thirty (30) of 30 medical records requested for review were received, and 19 
records resulted in validated hybrid hits.” 
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Toes 215 W. Pershing, Suite 600 
Kansas City, MO 64108 
FAMILY (816) 855-1888 * (800) 347-9363 
HEALTH ($16) 855-1890 Fax 


May 19, 2006 


Behavioral Health Concepts, Inc 
ATTN: Amy B. McCurry, Esq., MHSA 
2716 Forum Bivd., Suite 3a 
Columbia, MO 65203 


Dear Ms. McCurry: 


Children’s Mercy Family Health Partners is mailing this response to your email and draft documents dated 
May 15, 2006 and May 18, 2006. I have enclosed a copy of the documents I’m referencing for your 
convenience. Please note the following areas that need updates: 

May 18, 2006 Draft, page 6, Final Audit Rating, sentence 3. Please change the text to read that the 
Childhood Immunization Status measure was Substantially Compliant as there was not a significant bias 
associated with the overestimate. This will reflect the updates noted in the text of the document as reworked 
after our teleconference on May 17, 2006. 

May 15, 2006 Draft, Final Performance Measure Validation Worksheet: HEDIS 2005 Well-Child 
Visits in the First 15 Months of Life. I did not receive in the May 18" draft an update of this worksheet. 
Please update the table with the corrected Estimate of Bias (per May 18" draft text, page 5, 0.18% 
overestimate) and Audit Rating (per the May 18" draft text, page 6, Table 2, Substantially Compliant) 


Thanks again for your collaboration on this project. 
Please call me for questions or concerns at 816-855-1828. 


Sincerely, 


Jenny ey, RN, , CPHQ 
QM Manager 


Enclosures 


Group 
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Community CarePlus 


CA,RE| PLUS 
May 19, 2006 






Amy McCurry, Esq. MHSA 
Behavioral Health Concepts, Inc. 
Victoria Park, 2716 Forum Blvd. 
Suite 4 

Columbia, MO 65203 


Dear Amy, 


This letter is in response to the 2005 External Quality Review Performance Validation 
Protocol findings for Community CarePlus. Because immunizations are part of a well 
child visit, the score for Childhood Immunizations should co-relate to the score for Well- 
Child Visits. The final audit rating for Well-Child Visits was found to be fully 

compliant, yet the final rating for Childhood Immunization was found to be not valid. 


Reasons for the result in the Childhood Immunization results may be due to the tight 
schedule allowed for administration of immunizations. Missed appointments may cause 
delay in administration of the immunization causing the event to fall out of the timeline. 
Community CarePlus requests to review the thirty (30) medical records that were used in 
the audit. Please let me know if you can send the records by fax to 314-432-9295 or mail 
to me at 10123 Corporate Square Dr., St. Louis, MO 63132. 


Community CarePlus has incorporated immunization events from the State Public Health 
Immunization Registry (MOHSAIC) into the 2006 HEDIS report. 


Please call me at 314-432-9295 with any additional questions. 
p An 
“ean & Lane 


Sue Danis, RN, CPHQ, MHA 
Manager, Quality Improvement 
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HealthCare USA 


Crave CARE USA 


A foventry Health Core Plan 





May 22, 2006 


VIA FACSIMILE & U.S. MAIL 


Amy McCurry, Esq., MHSA 
EQRO Project Director 
Behavioral Health Concepts, Inc. 
2716 Forum Blvd, Suite 4 
Columbia, MO 65203 


RE: 2005 External Quality Review Performance Measure Validation Protocol 


Dear Ms. McCurry: 


I am writing on behalf of HealthCare USA of Missouri, LLC (“HCUSA”) regarding the 2005 
External Quality Review Performance Measure Validation Protocol (“EQR Protocol”) performed 
by your company, Behavioral Health Concepts, Inc. (“BHC”) as the external quality review 
organization (“EQRO”) for the Missouri Medicaid Managed Care Program (“Missouri MC+ 
Program”). We are in receipt of your May 9, 2006 report and have significant concerns with 
your findings. As explained in greater detail below, while HCUSA appeared to score well in the 
tested areas, and your report stated only a few areas for improvement, HCUSA’s overall 
performance measure validation review was deemed “Not Valid” due to HCUSA’s data not 
having been provided to the EQRO “in the format requested.” Specifically, your report stated 
that “data cannot be truly validated according to the CMS Protocols if it is accepted in a format 
that has already been processed by a software program.” Based on HCUSA’s commendable 
performance in the EQR Protocol in previous years, using the same processes and software as it 
did for this most recent audit, BHC’s findings come as a surprise, and do not seem justified. 
Further, the EQR Protocol report contained factual inaccuracies that HCUSA would like to clarify 
for the record. 


As you are likely aware, HCUSA uses the Catalyst Technologies (“Catalyst”) Quality Spectrum 
software for its HEDIS data collection and processing. Quality Spectrum is a SAS-based 
application. Each month, data is extracted from HCUSA’s internal data warehouse and loaded 
into Quality Spectrum. Quality Spectrum is NCQA-certified. Upon receipt of the results 
generated by Quality Spectrum, HCUSA provides such data to BHC for its review. This is the 
same process that was in place last year when HCUSA received a “fully compliant” review from 
BHC’s EQR Protocol. Please see enclosed copy of last year’s EQR Protocol report. Further, 
HCUSA also uses an NCQA-certified Auditor through HealthcareData.com, LLC, that audits the 
entire HEDIS process. 


BHC’s report faults HCUSA for BHC’s inability to “independently validate the denominator files 
because enrollment history and dates were not supplied to the EQRO for the members in the 
denominator files. Additionally [HCUSA] did not provide service codes that could be validated 
against the HEDIS Technical Specifications.” As was explained to the BHC reviewers who 
performed the EQR Protocol audit, HCUSA was unable to provide the enrollment history and the 
service codes because Quality Spectrum does not typically provide the data in the format in which 
BHC requested it. HCUSA staff indicated that the data could be obtained in the requested format 
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_2- May 23, 2006 


but more time would be needed to complete the different formatting. Instead, with BHC’s 
consent, HCUSA, as it did last year, provided BHC with a detailed demonstration of the Quality 
Spectrum software. BHC never expressed any dissatisfaction or problems, nor conveyed to 
HCUSA that the HEDIS data and/or Quality Spectrum software was insufficient or deficient in 
any way. If BHC had done so, HCUSA would have certainly worked expeditiously to address 
any concerns. Furthermore, because the same demonstration and discussion had taken place last 
year, HCUSA had no reason to believe that its protocol would not be similarly sufficient this year. 
Even during the closing conference between BHC and HCUSA, there was no mention of 
invalidity of any measures. 


BHC’s report also indicates that “the one criteria that was not met involved “detailed computer 
queries, programming logic, or source code.” Again, during the demonstration, HCUSA staff 
explained Ouality Spectrum’s functionality in this regard, and no questions or concerns were 
posed by BHC. HCUSA further explained at that time that the Quality Spectrum source code was 
proprietary to Catalyst, and that HCUSA could not provide it to BHC. 


As stated above, the Quality Spectrum software is NCQA-certified and approved. Interestingly, 
BHC does not utilize NCQA-certified HEDIS auditors or software in conducting the EQR 
Protocol audits. Moreover, HCUSA has reason to believe that BHC does not even follow HEDIS 
specifications. For example in the Well Child Visits in the First 15 Months of Life measure the 
EQR excluded members for not turning 15 months old in the measurement year. The technical 
specifications for this measure state: “Calculate the 15-month birthday as the child’s first 
birthday plus 90 days.” It appears that children who BHC excluded, in an attempt to validate this 
measure, had a birthday of October 1" and 2" indicating they would have been 15-months by 
December 31, 2004. Given that the Quality Spectrum software and the HealthcareData.com 
auditor are NCQA-certified, it would seem that BHC should defer to HEDIS rates generated by 
Quality Spectrum and validated by HealthcareData.com. 


Indeed, federal regulations support this concept, and have enabled states to use information about 
MC+ organizations obtained from private accreditation reviews, such as NCQA, to provide 
information otherwise obtained from mandatory activities such as validation of MCO 
performance measures. See 42 CFR 438.360, “Non-duplication of mandatory activities.” 
Specifically, 42 CFR 438.360 states, “to avoid duplication, the state may use, in place of a 
Medicaid review by the State, its agent, or EQRO, information about the MCO or PIHP obtained 
from a Medicare or private accreditation review” subject to the following conditions: 


(a) the MCO is in compliance with standards established by CMS or a national 
accrediting organization and such standards are comparable to standards established by 
the state to comply with § 438.204(g) and the EQR-related activity under § 
438.358(b)\(3): 


Response: HCUSA complies with NCQA standards, which are comparable to 
standards established by the state of Missouri. Further, HCUSA’s NCQA-certified 
auditor Final Audit Report (copy enclosed) states that “measures . . . submitted 2005 
Performance Report were prepared according to the HEDIS 2005 Technical 
specification and present fairly, in all material respects, the MCO's performance 
with respect to these specifications." 
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(b) compliance with the standards is determined either by CMS or its Medicare 
contractor, or a private national accrediting organization that CMS has approved as 
applying standards at least as stringent as Medicare; 


Response: UHCUSA complies because we use an NCQA-auditor who imposes 
standards pursuant to NCQA and such standards are as stringent as Medicare’s 
standards. 


(c) the MCO provides to the state all the applicable reports, findings, and other results of 
the Medicare or private accreditation review, and the state provides the information to the 
EQRO; and 


Response: HCUSA is obligated by 19 CSR 10-5.010, ef seg. to use NCQA-certified 
auditors, software, and processes, which HCUSA has done. When HCUSA submits 
its HEDIS data submission tools (“DSTs”), HCUSA is unable to make any changes, 
as the tools are locked by the auditors, and provided to the Department of Health 
and Senior Services and the Division of Medical Services as they were completed by 
the auditor. HCUSA has previously provided the state-required DST along with the 
full NCQA DST and a copy of the Final Audit Report from Healthcare Data.com, 
LLC. 


(d) in its quality strategy, the state identifies the standards for which the EQR will use 
information from Medicare or private review accreditation reviews, and explains its 
rationale for why the standards are duplicative. See 42 CFR § 438.360(b). 


Response: Again, 19 CSR 10-5.010(3) requires health care plans such as HCUSA to 
submit documentation from a NCQA-licensed organization that the quality 
indicator data have been audited according to HEDIS specifications. HCUSA 
satisfied this. Further, Table B of the regulation states that “data reported for each 
of the indicators . . . shall conform to the NCQA HEDIS Data Submission Tool and 
all other HEDIS technical specifications for indicator descriptions and calculations.” 
Additionally, in discussing Member Satisfaction Reporting requirements, which are 
a subset of HEDIS, HCUSA’s contract with the Division of Medical Services states 
“to reduce duplication and ensure consistent survey methodology, the state agency 
shall rely upon the member satisfaction survey data from this process. The health 
plan shall submit member satisfaction data to the Department of Health and Senior 
Services in accordance with 19 CSR 10-5.010, as amended. 


HCUSA’s use of Catalyst, and Quality Spectrum and as NCQA-certified software, should satisfy 
BHC’s need to validate HCUSA’s HEDIS measures, not only because of the federal regulations 
supporting such deference, but more importantly because BHC itself is not NCQA-certified. 


Finally, we request that BHC correct the various factual inaccuracies in the EQR Protocol report 
prior to issuing final findings. The following is a list of those inaccuracies: 


e Page 2: Catalyst Technologies, not McKesson, is the software vendor. 


e Page 3: The rate for HCUSA’s HEDIS 2005 Annual Dental Visit measure was 28.89%, 
not 28.79% 
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e Page 4: The rate for HCUSA in Western Missouri Region for the HEDIS 2005 Annual 
Dental Visit measure was 21.77%, not 29.77%. 


e Page 5: In Central Missouri Region, 1,166 eligible members were reported for the 
Childhood Immunization Status, not 412 members. In Eastern Missouri Region, 5,783 
eligible members were reported, not 5,792. 


e Page 6: Across all three regions, 7,258 members were found for the Childhood 
Immunization Status, not 6,504 members. 


e Page 9: Recommendations: 


o 1. HCUSA did comply with the recommendation from 2004 to utilize the Hybrid 
methodology in 2005 for childhood and adolescent immunizations. BHC was 
made aware of this during the EQR review. 


o 2. HCUSA did comply with the recommendation from 2004 to integrate HEDIS 
rate documentation procedures into corporate IS policies. BHC verbally 
recognized this integration during the interview process. 


Your prompt and thorough attention to this matter is greatly appreciated. HCUSA values greatly 
its perceived and actual compliance with state and federal requirements, and would like the EQR 
Protocol to accurately reflect such compliance. I can be reached at (314) 444-7237 if you have 
any questions. 


Sincerely, 


Reg uclinc. Probe 
Jacqueline Inglis 
Vice President Health Services 


HealthCare USA of Missouri, LLC 


Attachments. 


cc: Ms. Judy Muck, Assistant Deputy Director, MC+ Managed Care, Missouri Dept. of Social 
Services 
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Missouri Health Care Plan 


May 19, 2006 


Amy McCurry, Esq. MHSA 
EQRO Project Director 
Behavioral Health Concepts, Inc. 
2/716 Forum Blvd., Ste. 4 
Columbia, MO 65203 


Re: 2005 External Quality Review Performance Measure Validation Protocol 
Dear Ms. McCurry: 


We have reviewed Behavioral Health Concepts' summary of the External Quality Review 
Organization (EQRO) findings of the 2005 Validation of Performance Measures Protocol 
for Missouri Care Health Plan. As permitted by Option | of the Protocol, we are 

submitting comments regarding several of the findings and the review process as a whole. 


First, we would like to address our concern with the process for reporting of the results of 
the hybrid medical record review. From the information that Missouri Care received in 
the draft report, we are not able to respond to the accuracy of the EQRO's assessment of 
23 of 30 and 23 of 27 Childhood Immunization Status Combination #2 and Well-Child 
Visits in the First 15 Months of Life respectively meeting the EQRO's standard for a 
positive hit nor are we able to learn from our mistakes and implement improvements for 
next year. Missouri Care recognizes that the EQRO is unable to provide the health plan 
with the specific records that did not meet the EQRO's standards for medical record 
review, but the health plan would benefit from a general description of the types of errors 
the health plan made during medical record extraction in order to prevent similar error in 
subsequent years. 


Second, we want to address the areas for improvement and recommendations outlined on 
page 7 of the draft report. Below is the original EQRO recommendation, followed by 
Missouri Care's response: 


Areas for Improvement 

1. EQRO Comment - Missouri Care used a medical record review software, which 
was used as a tool for retrieving medical records that needed to be reviewed. This 
application has a module for data entry that does not indicate whether 
administrative data has been previously entered. The design of this module and 
the requirement to re-enter data leaves open the possibility for data entry error of 
valid administrative data. Missouri Care follows the process identified by the 
NCQA auditor, MEDSTAT. 
Missouri Care Response - The software does not require that Missouri Care re- 
enter administrative data. Missouri Care staff had inadvertently re-entered the 
data in 2005. The Missouri Care HEDIS staff have been trained and the problem 
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has been resolved in 2006. 


2. EQRO Comment - Documentation on HEDIS rate calculation and policies 
related to calculation of HEDIS measures are weak at Columbia office. 


Missouri Care Response - Missouri Care's HEDIS rates are calculated in 
accordance with NCQA's HEDIS Technical Specifications. Missouri Care's 
Manager of Quality Management, Sr. Quality Management/HEDIS Coordinator, 
and Quality/HEDIS Nurse all have a copy of the current HEDIS Technical 
Specifications. The technical specifications are reviewed for updates each year 
when the new Technical Specification are released and then used as a resource 
throughout the data collection and calculation process. Furthermore, current 
HEDIS policies are being revised and updated based on the EQRO's 
recommendations. 


3. EQRO Comment - QMACS uses 2 systems with 2 different member ID 
numbers, at present Missouri Care ends all [D's with an "n" for "new" or an "o" 
for "old" when the system is upgraded, this does not seem like the most effective 
way to ensure the uniqueness of ID numbers. 

Missouri Care Response - Over the years, new versions of the QMACS claims 
system have been released. Claims processed with older versions of the software 
are kept in separate databases. Schaller Anderson IT could not guarantee that the 
system ID for a member would remain the same from one version of QMACS to 
the next. Nor could they guarantee the system ID for a member on an older 
version wouldn't be reused on the newer version. When calculating HEDIS rates it 
is necessary to access both the claims database for the latest version of QMACS 
as well as the claims database for the older version. In order to ensure that a 
unique identifier is created for each member, APS creates an ID Bridge table. 
This table lists the member, his or her ID under the new version of QMACS and 
his or her ID under the old version. Their software then chooses one of these Ids, 
appends either an N (new version) or an O (old version) and uses this new 
moniker as the unique ID for that member in all of their calculations. 


4. EQRO Comment - The HEDIS end products are housed at the MOCare office in 
Columbia, but the staff at that office are still struggling to understand the inputs of 
the HEDIS calculations. 

Missouri Care Response - Missouri Care staff are confident in their knowledge 
of the rate production process. Based upon the 2004 EQRO report, Missouri Care 
has taken steps to not only understand the rate production process, but also to 
perform checks on the rates produced by APS. For example, Missouri Care's Sr. 
Quality Management Coordinator attended four days of NCQA sponsored HEDIS 
training in September of 2005, Missouri Care checks a random sample of hybrid 
records from APS to verify that all data sources were incorporated into the results, 
and Missouri Care staff calculate hybrid rates for comparison to the rates APS 
populates on the DST. 
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Recommendation 
1. EQRO recommendation - Improve documentation of the HEDIS rate calculation 

process by developing and maintaining a set of information system policies for 
the HEDIS rate production on-site to allow for continuity and validity of the 
process of rate production in the event of turnovers. 
Missouri Care response - Quality management staff will work on producing a 
HEDIS process flowchart, as well as process desktops, that clearly outline the 
entire HEDIS rate production process. 


2. EQRO recommendation - Increase ownership and control by MOCare over the 
process of calculation of HEDIS measures by designating an employee to closely 
coordinate with APS and clearly assigning responsibilities related to the HEDIS 
rate calculation processes within the organization. 

Missouri Care response - Missouri Care's Senior Quality Coordinator 1s 
responsible for coordinating the HEDIS rate production and has coordinated with 
Missouri Care's corporate office and APS in the rate production process 
throughout 2005. 


3. EQRO recommendation - Conduct and document statistical comparisons on 
rates from year to year. 
Missouri Care response - Missouri Care supplied BHC with documentation of 
statistical comparison's of rates from 2004 to 2005. Documentation was supplied 
in the original submission and on-sight. Please note that on page 6 of Missouri 
Care's report, the EQRO lists statistical comparisons of rates as a strength for 
Missouri Care. 


4. EQRO recommendation- Training of MCO staff involved in the oversight of 
coordination of performance measure calculation is strongly recommended. The 
NCQA offers workshops on the calculation of HEDIS measures. 

Missouri Care response - Missouri Care is committed to improving its HEDIS 
process. To that end, the Senior Quality Coordinator attended training offered by 
NCQA in September of 2005 and will attend additional training in the fall of 
2006. 


Finally, please note the following minor editorial comments on the 05/15/06 draft report: 
e On page | under "Interviews", 2" sentence it states that "This group was partly 
responsible for the process of calculating..." 
Response: The individuals listed above this sentence are responsible for the entire 
process. 


e On page 2 under "Findings", 2" and 3™ paragraphs the EQRO reports 95% CI for 
CIS as 52.56% to 74.76% and for WC15 63.16% to 88.22%. 
Response: It is unclear where the reported confidence intervals were obtained. 
Missouri are reported a 95% confidence interval of 58.88% to 68.44% for CIS 
and 70.57% to 80.82% for WC 15. 


Penning Performance Management Solutions Group 
hee A division of Behavioral Health Concepts, Inc. 568 





Missouri MC+ Managed Care External Quality Review Appendix 10 
Report of Findings — 2005 MCO Comments on Draft Report 


e On page 3 under "Documentation of Data and Processes" the EQRO states "The 
application has received NCQA certification and has been reviewed by the 
NCQA-certified auditor, MEDSTAT" (see also page 6, item 6 under "strengths"). 
Response: NCQA not MEDSTAT reviews the source code. 


e On page 6, item 5 under Strengths" states "Austin Provider Solutions (APS) 
provides the members' identification data to the SPHA and obtains an extract of 
immunizations for the members eligible." 

Response: Missouri Care staff, not APS, sends the SPHA a list of eligible 
members to obtain an extract of immunizations for the member's eligible. The list 
is then returned by the SPHA to Missouri Care who sends the list to APS. 


As we look to strengthening Missouri Care's Quality Management program, we welcome 
your recommendations for improvement and we appreciate this opportunity to respond to 
the review process and the findings of the EQRO. Should you have any questions, please 
contact me at 1-800-322-6027 ext.4623. 


Sincerely, 
| ee EN le XQ 


Tammy Wiese 
Manager, Quality Management 
Missouri Health Care Plan 
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